Yes. Your W4 has no affect on your filing status.
If you fill out your W4 as Married filing separately with one child, only one of you should include the child. Otherwise, you will both have less withheld based on that child. Although the credit is only $500 for your child over 17.
How you fill it out depends on your goal, your income, your spouses income and your overall tax situation. If you want a refund, this is slightly more likely to happen if you both do fill out the W4 as Married Filing Separately, then actually when you file your return file jointly. You may want to use a W4 withholding calculator to help you decide.
What Is a W-4 Form?