The question I would ask is, can you prove that you requested the $2500 deposit to be for tax year 2019? Do you have a receipt or other proof that this was supposed to be a 2019 deposit?
If you can prove this was supposed to be a 2019 contribution, and it is the bank's error to record it for 2020, then I think you have a good argument that the bank should correct it. If they refuse, I would escalate to a higher employee level. You can also complain to the bank's regulator. (It might be the state banking department if this is a bank, or it might be the SEC if this is a brokerage.) If in the end, the bank won't correct its mistake, I suppose your last recourse is to move your IRA to a different bank.
If this was your mistake, and you forgot to select "2019" as the tax year for the contribution, that's on you and there is nothing to be done. You need to amend your tax return for 2019 and pay the penalties. It is too late (in 2021) to remove an excess contribution for 2020.
And in the future, you probably should check your contribution record at the IRA after the deposits are reconciled (a week or so) to make sure they are recorded in the way you expected.
