When I did my brothers taxes, my family did not tell me I had to file him under my mom as a dependent of hers. I messed up and need to know how to fix.

You cannot change or add anything on the return that you just e-filed, nor can you stop it.  It is too late, just like when you put an envelope in a US mailbox on the corner.

Now that it was accepted there is nothing to do but let the IRS process it fully.   When the Form 1040X becomes available at the end of March, you will need to amend your brother's return.  Do NOT try to change anything now.  Wait.

 Your mom will not be able to e-file.   She will have to print, sign and mail her return that claims your brother as a dependent.   Even if you amend his return later, the amended return never "catches up" in the system so she cannot e-file this year.