I currently have a full time job and part time job. How could I adjust my w-4 accordingly to avoid paying back taxes when I file next year?

You should utilize the IRS online withholding estimator.  The estimator will tell you how to complete a new W-4 for one or both jobs so that you don't under withhold and owe a lot when you file next year.  IRS Tax Withholding Estimator

Go ahead and fill out both job's W-4 forms as if you held just one job.

BUT

For the full-time job, enter some $$ in line 4c for some extra withholding....an amount sufficient to make up the shortfall you experienced this year.....thus, if you put $50 on line 4c, and get 26 pay periods per year, that would increase your withholding by $1300 for the year.  (Though, we're already into March, so there's fewer pay periods left....but you can figure that out according to how many you have left....and your change may not take effect for a couple weeks, so could miss one)