Should RSUs sold to cover tax be subject to capital gains tax if they were only sold to cover taxes in the first place? They never were before and I did not cash out.

It depends whether the RSUs "sold to cover" were subject to capital gains tax.  If the RSUs that were sold to cover the withholding were sold at a price greater than the price assigned on the vesting date, then yes, you have a capital gain, and such gain is subject to a short-term capital gain tax.

Ordinarily the sell to cover is done at the same time as vesting so that there is no price movement in the underlying stock.  However, we don't know how long your company waited until selling the RSUs for withholding purposes.  Had they waited some period of time, and the underlying stock increased in value, then when those shares were sold to cover, you would have a short term capital gain.  

If you can provide the numbers regarding the tax, and the value of the RSUs that vested, and the amount that were sold and the gain, that would help us to better understand your tax situation.  

@murreweet