Multiple jobs, and spouse increased income

I currently have no federal taxes coming out of my paychecks. I filled out the w-4 as suggested, and I am wondering if it may be because me and my spouse have two dependents? I also have a second job where I make less money at. That section was confusing to fill out. My husband is starting a new job next week and it will be a six figure income. We are very nervous about filling out the w-4, and owing the government next year at tax time. Please help. Thanks!