Normally an annuity defers the income taxes from being paid and once it is taken as a distribution it is taxed.
Did you take money out? Did they issue you a 1098INT?
"If you buy the annuity with pretax money, then the entire balance will be taxable. If you use after-tax funds, however, then you'll be taxed only on the earnings. If you cash out a deferred annuity in a lump sum, then you'll have to pay income taxes on all of the earnings higher than your original investment."
Source: http://www.kiplinger.com/article/insurance/T003-C001-S001-how-annuities-are-taxed.html