Hi jaawarne,
Thanks for the question. So your wife won't have withholdings going forward. You don't have to update your form - it will likely result in higher withholdings than necessary. If you do choose to update you would contact your HR/payroll department to fill out a new form. It might help if you could get a copy of your old W4 to compare. Intuit's W4 calculator may come in handy. W4 Calculator
On the new W4 form make sure you fill out step 3 to include all your children under 17. Other than your basic info in step 1 that should be all you need to fill out. I hope this helps.