As you propose, to correct the excess Roth IRA contribution from 2021 and avoid a penalty for 2024, you must before the end of 2024 (completed today or tomorrow) obtain a regular distribution from your Roth IRA(s) of exactly $6,000, no adjustment for investment gain or loss. Tell the Roth IRA custodian nothing about this distribution being to correct an excess contribution. This distribution will be a nontaxable distribution of contribution basis reported on your 2024 tax return.
With regard to penalties, you owe a 6%, $360 penalty for each year 2021, 2022 and 2023 (and 2024 if you fail to obtain the corrective distribution in the next two days). You'll pay these along with filing 2021, 2022 and 2023 Forms 5329.
An alternative to having the 2021 contribution be an excess contribution might be to amend 2021 to Married Filing Jointly. You have until April 15, 2025 to do that, if it makes sense.
