Submitted the wrong forms and ended up paying more in taxes

Yes, you should.  If you are amending your return, you need to send the corrected 1040NR attached to the Form 1040X.  They would need to be mailed.  

The refunds will not be the same on the nonresident returns you will need to file.

You can use the steps here below to amend from 1040 resident return and file a correct 1040NR (nonresident return).    You will need to have your correct tax return prepared as a 1040NR.   Please visit the TurboTax/Sprintax site for guidance.  

Print your 1040NR in Sprintax; do not e-file it..  You must mail the 1040NR along with the amended return Form 1040X. 

To amend and file the correct return, do the following:   

1. Enter your name, current address, and social security number (SSN) or individual taxpayer identification number (ITIN) on the front of Form 1040X.  
2. Don’t enter any other information on page 1.  Also, don’t complete Parts I or II on page 2 of Form 1040X.  
3. Enter in Part III the reason why you are filing Form 1040X.   
4. Complete a new or corrected return, Form 1040NR,  
5. Across the top of the new or corrected return, write “Amended.”   
6. Attach the new or corrected return to the back of Form 1040X.  

**Make sure you check the box to show the tax year that you are amending on Form 1040X.   

Form 1040X  

1040X Instructions