All of the above.
If you have excess contributions for 2023 and 2024, you must each report those excess contributions and pay the 6% penalty. If you did not report the excess and pay the penalty, you must each file your own amended returns for 2023 and 2024 to include form 5329 and pay the penalties.
Then for 2025, it is too late to use any kind of special procedure to remove the excess, so you have to pay the penalties again. Each of you will file a separate form 5329 for your personal situation (your joint return will include two copies of form 5329, one in each spouse's name.)
Then sometime during 2026, you can make a regular withdrawal of the excess contributions. You do not have to withdraw the earnings, just the contributions. This is a regular withdrawal, not a special withdrawal of excess contributions. Because it is a return of contributions, it won't be taxed. Then you will include two copies of form 5329 on your 2026 tax returns to remove and resolve the penalties.
Two additional notes:
1. If you file jointly for 2025, and you made allowable Roth IRA contributions, you have the option to remove them, and then your 2025 limit can be applied to the excess carryover from 2023 and 2024. For example, if you have $13,500 of excess contributions, and you are eligible in 2025 but make no new contributions, you can apply $7000 of the excess toward your 2025 limit, meaning that the 2025 penalty will be figured on the remaining $6500 of carryover excess. Then in 2026 you can either withdraw that excess or use it up by applying it to your 2026 limit.
2. If you would be eligible if you had filed jointly, you could amend your MFS return to MFJ. This would make the contributions allowable (probably) and there would be no penalties owed. The IRS will allow you to amend from MFS to MFJ. However, I presume there was a good reason for filing separately, so you should consider whether these penalties offset the other benefits of separate filing. Amending from MFS to MFJ requires special instructions, let us know if you want more information, and if you used Turbotax for those years or not.
