No, this does not sound right. As a resident of Virginia, Virginia has the right to tax all of your income. Since your employer was not collecting Virginia state taxes, you most likely have a large amount owed there, possibly including underpayment penalties.
As far as California is concerned, they consider the source of the income to be the location of the employee when the services are performed. If you did not ever set foot in California, you are not subject to their income tax. You should be getting a full refund of all taxes that were paid to California. Furthermore, you will want to notify your employer to either start withholding Virginia state taxes, or if they are not able to do that, at least file an exemption form for California state taxes and start paying estimated taxes on your own to Virginia.
