Refunds of federal taxes are never taxable income. If the IRS paid interest on the amount refunded, the interest only is taxable income.

Refunds of state and local income taxes may be taxable income. 

If you took the state and local tax deduction in the year of the refund and itemized, the refund will be taxable in the year received if the deduction benefitted (reduced) the prior year's tax bill. This is true even if you opted to apply the refund to this year's state or local return.

But if you took the sales tax deduction or the standard deduction in the year that caused the refund, it won't be taxable on this year's return. It also won't be taxable if the prior year's return was subject to the AMT (Alternative Minimum Tax) and the refund amount was less than the amount disallowed under AMT.