Contact your Human Resources (HR) department. Apply your answers based on your filing status, and if applicable: spouse income and number of dependents you will claim at the end of the tax year.
If married, both should not be claiming the same dependents ( ex 2 children and both are claiming 2) on the W4 as the income is combined at the end of the year and there is only 2 children on the entire return and not 4.
If you have online access, you may be able to update your form W4 there without contacting HR.
TurboTax has a tool that helps you to calculate your withholdings. Here is the link.
https://turbotax.intuit.com/tax-tools/calculators/w4/