After you file

Did you file a Joint return?  .If you remove him you will need to file as MFS married filing separately.  You can only amend from Joint to MFS before April 15.  

Or how did you file?  If you didn’t file Joint why his ssn on your return?  If you filed MFS and show his name and ssn that won’t prevent someone from claiming him.