CA-source income is income from work actually (physically) performed in CA. If you never actually worked in CA, that income is not CA-source income.
EDIT: Due to a September 2019 court decision, the income of non-resident sole proprietors providing services to CA businesses is now taxable by CA, even if the sole proprietor never worked in CA.
https://www.coblentzlaw.com/california-office-of-tax-appeals-gives-precedential-authority-to-bindley...
