the law says excess contributions must be withdrawn by the due date of the return including any valid extensions. you're saying this was not done. therefore you need to file an amended return for 2021 with form 5329 and pay the 6% excess contribution penalty. it is also possible that without amending the IRS will send you a tax bill for the excess contribution penalty because they too get a copy of the 5498.
the 1099sa you'll receive in 2023 will show the excess HSA withdrawal
it's up to you if you want to pursue reimbursement by the bank for messing up your request for a timely withdrawal.
as an aside, you could have left the excess in and used it for your 2022 contribution. the penalty would still have applied.
