You should have filed Form 8889 in which you would have reported that your HSA withdrawals were for qualified medical expenses in which case they would not be taxed.
