Taxes paid in both 2018 form1040 and 2019 form1040 on Excess Roth contributions that were withdrawn.

Since these are Roth IRAs and the excess contribution was withdrawn prior to tax deadlines each year, shouldn't we be tax-exempt? If so, do I need to file amended returns for 2018 and 2019?