You can adjust the W-4 if you wish but it is not required. 

And if you have health insurance let them know so they can be added to the policy.  

If you do not adjust your W-4, you will probably get a refund that is $2000 larger than last year.  Possibly much higher if your income is low enough that you qualify for EIC.

If you would prefer to reduce your paycheck withholding and get a little extra every week instead of a big check at the end of the year, you need to submit a new W-4.  If you are married, then your spouse also needs to submit a new W-4, because of how the form was redesigned.  Get the full W-4, not just the signature page, and go through all the calculations that apply to your family situation, and then follow the instructions as to what you and your spouse should claim. 

You say after your daughter's "RECENT" birth, indicating to me that she was most likely born in the last quarter of the 2020 tax year. (congrats on the new addition to the family!).

I would recommend you DO NOTHING at this time with your W-4 at work concerning taxes. This close to the end of the tax year, you will find the extra $1,500 to $2,000 you'll save on your 2020 tax return will come in handy and probably be s short term financial life saver. Pamper's and baby formula are *EXPENSIVE* and I can assure you that you will ***NEED*** the money. No doubt about it. Especially around the Apr-May 2021 time frame when, in addition to pampers and formula, you have to start buying baby food.

Then in early 2021 once you've completed your 2020 tax return, you'll know your tax liability down to the penny and can make a more informed decision on changes to your W-4 for the 2021 tax year and beyond.

Note that for a child born in 2020, even if born one minute before midnight on new year's even, you get to claim that child as your dependent for the *entire* tax year. The $500 child tax credit is practically guaranteed if your income is not to high, plus the earned income credit and possibly the child care credit if you pay for any child care in 2020.

The only thing you really need to do at work right now, is make sure that baby is covered on any work sponsored health insurance along with adding her to any other employer provided family benefits packages that may be a part of your employment compensation.