It's possible that 6 allowances could be right for you. I'll explain some of the reasons below. At the same time, you want to have enough withheld.
In your situation, depending how much you earn for the year, you may be entitled to certain credits because you now have a child. The earned income credit is available when the income is below $40,320 for 2018. This credit is refundable so it is like you paid this in. The credit is provided to low income taxpayers who are trying, but struggling. The maximum credit is available at the mid range ($0-$40,320). As you earn money the credit increases and once you reach the mid range it begins to decrease. The maximum credit for one qualifying child in 2018 is $3,461 with one qualifying child.
In 2018, there will be a Child Tax Credit in the amount of $2,000 (doubled from 2017).
The IRS calculator can be used any time and you can complete a new W4 as many times as you think you need to throughout the year. So test it whenever you want.
https://ttlc.intuit.com/replies/3301783
