How do I know if the ESOP retirement plan I took an early withdraw from is tax exempt?

An ESOP is not tax "exempt".  They are tax deferred.  An Employee participant in an ESOP does not pay tax on stock allocated to their account until they receive distributions.  ESOP distributions are taxable the same way a traditional IRA or 401(k) is taxed.  Like other qualified retirement plans, ESOP distributions received by employees under age 59-½ (or, in the case of terminating employment, under age 55) are considered early withdrawals, so they are subject to normal applicable taxes, plus an additional 10% excise tax, which is commonly referred to as a penalty.  The excise tax can be avoided by rolling over the ESOP account balance into another qualified retirement plan (e.g. a 401(k) or a 403(b)) or a Traditional IRA.