If this job is your only source of income, you should be able to enter $4000 in part 3 of the W-4 (if you have 2 dependents under 17 years of age). The payroll system for the place of employment should be able to accurately withhold the right amount, based on your filing status and number of dependents. Sometimes it can be off if their are multiple sources of income in the household, but in your case it should be easy.
If you have a dependent child under 17 and do not earn a certain amount of income, there is a possibility that there should not be any federal withholding taken out. Without knowing your salary, there isn't a way to tell if they are taking enough federal withholding out. You would need to contact payroll.
