Federal and State Withholdings

Hello,

I currently have a second job which is part-time and noticed that no federal or California state withholds taxes have not been taking from my paycheck. I normally work 8 max 20 hours a month.  I'm worried that I will owe taxes.  My first job is full time and I make decent money and taxes are being withheld from both state and federal. Including that I am filing single.