Selling house lived in 7 years, purchased land and building new house, house sale gains use question?

So we have lived in our current and first house for 7 years. We purchased land in April of 2023 and financed half  of it. We have been working with a custom home builder to build our new house on that property. Because the land was raw land we have been paying cash to get the clearing, grading, septic, well and electricity done so that we can move into an RV on the property while the house is built. I am also looking at financing a shed to allow us to store stuff on site as well. 

The question is, with the profit we make from the sale of our house what tax implications do I need to know? From my understanding if you lived in the house 2 of the 5 years prior to sale and file married jointly you can use up to $500k with out paying gains tax. But can I use some of the profit we are going to get to pay off the land and shed and then put the rest into the build of the new house or does all the profit need to go to the build of the actual house? The profit from selling our house could be $190k but the total project (land, site work, shed, utility work, and house build) will be close to $450k. Another note, we are doing a private loan for the balance not payed by our house sale profit and cash. 

Thank you in advance!