If I pay state taxes to Maine where I live and work for all of 2016 why do I have to pay state taxes to virginia my resident state where I didn't work or live

You choose your residency status based on all the facts and circumstances, such as your family remained and you intended to return back home.  

If, as you indicate, that you never changed your residency, did not plan to change your residency and went to the other state for a temporary assignment (a job with a foreseeable end date) then follow the filing requirements here.  

Note:  Reciprocal agreement does not apply if the states are not physically adjoined.

• Nonresident State:  All income earned while working in this state are required to be taxed.
• Resident State: All worldwide income is taxed to your resident state.  If you pay tax on the same income to another state then you can choose to receive the "credit for taxes paid to another state".  Your resident state will credit the amount of tax paid to the other state or their own tax that would have applied, whichever is lower.  This eliminates double taxation on your income.

If you conclude that you moved and the state where you worked became your resident state, then file only that state return as a resident.