As @fanfare noted, if you contributed less than the maximum in any year since 2021, that can also be used to reduce the excess that you carry forward, that will all be accounted for on form 5329.
"I read that the IRS may waive the penalty in some cases if I provide a reasonable explanation."
I think that is unlikely here. What you would be thinking about is if you missed the 2022 deadline to remove the 2021 excess due to a family emergency or other situation. If you doubled your contribution for 2021 due to some confusion, a mixup with your finances, etc., you might be able to waive the penalty for 2021 if you have a good story, but I don't know if you can extend that for 5 years. Eventually you have to be responsible for your own actions.
To request an abatement of the penalty you would use form 843. You file this on your own, it's not included with a tax return. And you would not be able to file this until after you filed all the amended returns and pay the penalty.
https://www.irs.gov/forms-pubs/about-form-843
Note, that if you file amended returns showing that you owed $360 for 2021 (due April 15, 2022), $360 for 2022 (due April 15, 2023) and so on, and you pay those taxes, the IRS could come back and assess late fees and interest backdated to the date the payment was due. That would be another opportunity to request a waiver or abatement of the penalty.
My suggestion would be to pay the $360 for 2021-2024, wait and see if you get assessed late fees, and then apply for a waiver/reduction all at once. The IRS will usually forgive late fees for first-timers, but they can't waive the interest. I think you will need a good story to also get the base 6% penalty reduced. But good luck.
