Remember that the IRS and California treat it differently. So yes, you can withdraw money from your HSA as far as California is concerned and do whatever you want with it. But the IRS has different rules. So unless you withdraw the money for qualified medical expenses the IRS will tax it and levy a 20% penalty in addition.
