The interest, dividends, and capital gains earned by your HSA are not taxable at the federal level, but they are taxable in California and New Jersey.
If you entered them on your federal return in previous years, then you could amend those returns to recover the tax paid on the earnings. However, you should look to see how much money you would actually save, to judge if it is worth the cost and hassle.