First of all, I'm sorry for your loss. Regarding the tax credits, a child who is born and died during the same year is still eligible for the same tax treatment as though they'd lived, and resided with their parents, for the entire year. Per the IRS's qualifying child rules, "If you meet all of the other requirements to claim this credit, and your child was born and died in 2020 and didn't have an SSN, instead of an SSN, you may enter "DIED" on line 2 of Schedule EIC (Form 1040), Earned Income Credit and attach a copy of the child's birth certificate, death certificate, or a hospital medical record showing a live birth. "  Please note that a return would have to be filed by mail, not efiled.

What if they were just born in 2021. How would I get the monthly payments that just started this month?

You would receive advance payments by adding them in the Child Tax Update Portal. They will be adding that function in late summer, but you can set up an account there now so that you're ready to go when they do. However, if they don't have SSNs, I doubt if you'll be able to do it in the portal. In that case, you'll get the full amount of the credit added to the return you'll file this spring.

When you file the return next year you MUST follow the screens carefully  so you correctly answer they lived with you THE WHOLE YEAR ... failure to choose all year as you are told to do will result in your getting no credits.