A 43-year-old woman was referred with a 10 month history of persistent pain in the left orbit. Two years prior, she experienced similar pain in the right orbit. Magnetic resonance imaging (MRI) at the time revealed an enlarged right medial rectus muscle. She was diagnosed with idiopathic orbital myositis and was successfully treated with oral corticosteroids. A year later, she developed symptoms in the left orbit with similar imaging findings. For ten months, she remained on high dose corticosteroids for presumed left medial rectus myositis before presenting to our service. Computed tomography (CT) imaging after corticosteroid taper revealed enlarged left medial rectus and left lateral rectus muscles. Orbital biopsy established a diagnosis of granulomatosis with polyangiitis (GPA), for which she was successfully treated with rituximab. This case underscores the importance of not only proceeding with biopsy in atypical cases of orbital myositis but to also taper steroids prior to biopsy.