What is line 6 of part I of form 8379 exactly asking? If I made extra payments outside of what is taken out of my paycheck? Is what's taken out considered payment?

Actually, it's asking about both. 

Although the Instructions for Form 8379 don't provide line 6 instructions, line 6 simply wants to know whether you reported any withholding (from Box 2 of Form W-2, or other sources), or any estimated tax payments -- payments you've made throughout the year to the IRS -- on the tax return that relates to this Form 8379.

If you'll check out the structure of Form 8379, Part I is simply trying to determine whether you meet threshold requirements to claim the allocation. 

If you answer "Yes" to line 6, you go straight to Part II of the form, and then enter relevant income tax withheld and other payments in Part III, on lines 19 and 20, respectively.