Your employer "considered" you to be physically present in CA, but in fact you never were - according to your original post. Therefore your W2 income is not subject to CA income tax per California law. That's why your employer corrected your withholding in mid-year.
**Answers are correct to the best of my ability but do not constitute tax or legal advice.
