Yes, you need to pay CA tax on the income you received for the days you actually worked in CA. So, you should probably file a non resident return. See http://money.cnn.com/2013/06/28/pf/taxes/business-traveler-tax-threat/ Since you live in a state without an income tax (WA), it’s more likely you should file in the work state. You can't use the "it all comes out even" rationale for not filing.
Your employer is not required to withhold for CA, but may be willing to do so. Since the amount you owe will probably be small, you may want to forego withholding.