Failed to use annualized income method and received a penalty

I used Turbo tax to file my taxes and received a penalty for not sending estimated taxes the first three quarters even though taxes were withheld on income for the first 3 quarters.  I failed to use an annualized income method.  I did a 401k rollover to Roth in December and paid the estimated tax on the rollover in the 4 quarter on time. I filed a 1040x but have received a notice that the “IRS can not change the tax penalty because it is based on the tax shown on the original return”.
I overpaid my total tax liability and got a refund.   Is it legal for the government to keep the penalty money if I did an amended return of my taxes?  
Thank you.