I'm a graduate student who goes to school in Pennsylvania. Do I need to file state taxes for Pennsylvania?

Under the terms of a reciprocity agreement between New Jersey and Pennsylvania, compensation paid to Pennsylvania residents employed in New Jersey is not subject to New Jersey Income Tax under the terms of the Reciprocal Personal Income Tax Agreement between the states. Similarly, New Jersey residents are not subject to Pennsylvania income tax either. Compensation means salaries, wages, tips, fees, commissions, bonuses, and other payments received for services rendered as an employee.

If you are a New Jersey resident and your employer withheld Pennsylvania income tax from wages, you must file a Pennsylvania return to get a refund. To stop the withholding of Pennsylvania income tax, complete Form REV-419EX, Employee’s Nonwithholding Application Certificate, and give it to your employer. 

Please see this New Jersey webpage for more information.

See also this TurboTax article and this one on multiple-state situations.

Q. Do I need to file state taxes for Pennsylvania?

A. No, because of the reciprocity agreement and the fact that no PA state tax was withheld (so, you don't need to file for a refund of PA state tax).

The reciprocity agreement does not apply to city tax.  So, your employer was correct to withhold city tax.