Firstly, your allowances don't change your overall tax bill, just who pays how much in advance, and how big a refund you get.
Generally the IRS advice for 2-earner couples is that you calculate the total number of allowances on a joint basis, using the worksheets on both sides of form W-4, and then the higher earned claims all the allowances and the lower earner claims zero.
If you planned to file married filing separately you would want to be more balanced.
You can also use the IRS withholding calculator here to get an estimated of your tax refund or amount due, and some recommendations on how you can change your withholding. https://www.irs.gov/individuals/irs-withholding-calculator
For 2018, if you add a child born in November, and assuming you and your spouse both earn the same as your did last year, you would expect a $2000 larger refund than last year. To get that money back in your regular paychecks instead, one of you could add 4 allowances for the rest of the year (then back it down to 2 extra allowances in January). But that's just a rough estimate and if one or both of you take time off and reduce your income, that will affect the calculation as well.
