When you file your taxes, you get to claim the taxes withheld through backup withholding as payments you made, just like withholding from your paychecks. Wasn't it listed on the 1099-INT that the bank sent you each year?
https://www.irs.gov/taxtopics/tc307
