In general, married will withhold less than single, and each dependent you claim will reduce your withholding further.
Use the IRS withholding estimator.
https://www.irs.gov/individuals/tax-withholding-estimator
Remember that you run the calculator one time with all your combined income and deductions, and the calculator will give you adjustments for both spouses, you need to make the adjustments together for it to work correctly. You should also plan to run the calculator in January to set your withholding for 2025, since any mid-year adjustments you make now might not be appropriate for a full year.
You can also run the worksheets on the paper W-4, be sure to follow all the instructions on all 4 pages.
https://www.irs.gov/pub/irs-pdf/fw4.pdf
The default answer would be for both spouses to mark "married filing jointly" in Step 1c, and both spouses check the box at 2c. The calculators can give you a more accurate result.
When both spouses work, having both spouses claim married status will usually result in too little withheld, because both spouses get full credit for the standard deduction when you only get one standard deduction on the tax return. So some adjustment is needed, but both claiming single is probably resulting in too much tax taken out. You will get that back as a tax refund when you file.
