By mistake my bank took Backup Withholding out of my savings account the last few years. Is there a way to get that money back?

yes you need to file an amended return for each year 2016-2018.   however, if the amounts are small, it may cost more to amend (TT fees) the you'll get back.  The IRS does not require you to amend in your situation.   any amounts for 2015 or prior are lost because the period to file amended returns for those years has expired.. 

When you file your taxes, you get to claim the taxes withheld through backup withholding as payments you made, just like withholding from your paychecks.  Wasn't it listed on the 1099-INT that the bank sent you each year?

https://www.irs.gov/taxtopics/tc307

To enter withholding on a 1099-INT, you enter it on the same screen
Federal Taxes tab
Wages & Income
Interest and Dividends
Interest on 1099INT - Click the Start or Update button

There is a little checkbox underneath for My form has info in more than one box
Click on that and more boxes will open up

I didn't know what it was, so I didn't claim the withholding tax. Does that mean I have to make an amended return?

Thanks. This year it's $100 withholding, so it may be worth it to do an amended return.