Do I mail my federal return in with NJ 1040-NR return

It depends, if you do; the instructions that print at the end with the return will guide you on what forms are needed to be sent.  

2024 Form NJ-1040NR

In most cases you do not include your entire federal tax return with your NJ-1040NR. The only case in which you would have to include your entire federal return is if you were a part-year nonresident, your income for the entire year was equal to or less than the filing threshold amount, and you are filing to get a refund.

There are some situations in which you have to enclose federal Schedule B, Schedule C, Schedule K-1, Form 2106, or Form 8283, but not the whole federal return. The details are in the Instructions for Form NJ-1040NR.