A W-4 is simply the way provided by the IRS for workers to estimate their income tax so the employer will withhold an amount that is close to the tax you owe. Your actual tax is only calculated on your tax return. If you owe more than was withheld, you must make a payment, and you may be subjected to penalties for being under-withheld during the year, even if you pay in full when you file.
You and your spouse should prepare new W-4s at both your jobs. Get one copy of the W-4, fill out all the worksheets (all 4 pages, not just the front page) and use the result to prepare both W-4s.
You can also use the IRS calculator here.
https://www.irs.gov/individuals/tax-withholding-estimator
