If someone is preparing your spouse's taxes ("his taxes") ---according to you----then you cannot also file a joint return. Your spouse cannot file his own separate return and file jointly with you as well. You need a much clearer understanding of exactly what your spouse---and the attorney --- are doing.
You do not say how long you have been married...were you married to him in 2017---or even earlier? What have you been doing about your tax returns while you have been married to him?
If your spouse owes a lot of back taxes---you might want to get some legal advice yourself to protect yourself from being held responsible for his debts.
Provide some details----someone here may have some helpful thoughts.
Are you just married in 2023? If not, how have you filed in the past when your husband didn't file?
You should speak with the attorney. In general, if you file a joint 2023 return, you will be equally liable for any income, tax, errors or omissions on that return (2023). It doesn't automatically make you responsible for past problems, but your "share" of any 2023 refund could be taken by the IRS to offset a previous tax debt. Whether you might already be responsible for past problems requires more details.