I recommend using the W4 calculator to generate W4's for both you and your spouse.
The link to the W4 calculator is here:
Where's My Refund? | Internal Revenue Service (irs.gov)
The tool will generate W4 forms for each spouse to give to their employer.
If you do decide to complete the W4 manually, I recommend that:
- You each claim single in Step 1c
- Only 1 of you claims the child in Step 3