If I make less than 6,000 a year and filing jointly with my husband who claims our son - do I withhold anything on a W-4?

No, the situation sounds like you would not need to have additional federal withholding from your w-2 job beyond medicare and social security. The standard deduction for married filing joint in 2021 is $25,900, so that will more than cover $6000 in W-2 wages, provided you do not have additional income.
To be sure, please do try our withholding calculator. You can plug in everything and if something changes later, you can check it again and see if you need to make any changes. This is a package of our free tools and the W-4 Calculator is toward the bottom, on the right. Please click below.
TurboTax Taxcaster and calculators