Out of State Employer

File a nonresident New Jersey and resident New York income tax return if your NJ-based employer withholds NJ income tax.

You do not say whether you are permanently working in NJ—or are doing so due to the pandemic.

If NJ tax was incorrectly withheld

It's possible you may not have to pay NJ tax if you were always a NY-based employee with an established home office. In that case you would file a NJ return to claim a refund. Do this by changing NJ wages to $0. Include a copy of the NY return and a signed statement saying NJ tax was withheld in error.

Since you would get all your withholding back, there would be no tax credit on the NY return.

If NJ tax was correctly withheld

Otherwise, you report your wages to NJ and claim a credit on the NY return as you mentioned.

NJ says: "New Jersey sourcing rules dictate that income is sourced based on where the service or employment is performed based on a day’s method of allocation. However, during the temporary period of the COVID-19 pandemic, wage income will continue to be sourced as determined by the employer in accordance with the employer’s jurisdiction. See Telecommuting.

I suggest speaking with your employer's HR or payroll department if you have questions about whether you should be paying NJ tax.