My employer paid grad school tuition this year. This was non-taxable income (a rare case). But forms were incorrectly filled out. How do I claim this withheld amount?

You don't. Your employer has to handle it on his end. UNLESS the box 1 amount of your W-2 is correct (and they just took out to much in box 2). Then you file normally and the excess will be refunded.

If you employer incorrectly reported it as taxable income, there is nothing you can do, on your tax forms, to undo that.

By law your employer can only give you $5250 maximum, tax free (it's not a rare case). If you got more than $5250, the amount above $5250 is usually already included in box 1 of  your w-2 and you do not need to enter anything additional on your tax return. If that's all you asking about; then it is already correct*.

But, since you have paid tax on that part, it  is considered your after tax money and that amount can be used  to claim the tuition credit.

*You used the expression rare case of not being taxable.  It is possible (but rare) for more than $5250 to be tax free. But, then it's not tuition reimbursement; it's business expense reimbursement. But again, you cannot correct your employer's mistake by claiming something on you tax forms (you could before 2018). He has to give you a corrected W-2.