@Jbrown6802
The simple answer to your question is: you do not have to let him claim any kids. It's all voluntary on your part, if you want to.
Neither a court order or notarized piece of paper has any standing with the IRS. They go by their own rules, which essentially say the custodial parent has first priority on claiming the children on her taxes; regardless of the amount of support provided by the non-custodial parent. The IRS goes by physical custody, not legal custody. The non-custodial parent can only claim the child as a dependent if the custodial parent gives permission (on form 8332). If he wants to enforce the "notarized piece of paper", he'll have to sue you, in family court.
"He argues constantly that he should claim one because there's a cap on how many kids you can claim". There's a grain (or 2) of truth in that. The Earned income credit (EIC)* is capped at three kids. BUT, the non custodial parent is not allowed EIC for kids, even when he gets a form 8332 to claim one as a dependent.
There is no limit on the number of kids that can be claimed for the child tax credit, but there is a mathematical limit on the amount of the child tax credit. So, depending on your income, you may not be getting anything** for the 4th kid and less than what he could get for the 3rd kid. You should do the math and see if the family would come out better releasing one or two kids to him.
*There is a special rule in the case of divorced & separated (including never married) parents. When the non-custodial parent is claiming the child as a dependent/exemption/child tax credit; the custodial parent is still allowed to claim the same child for Earned Income Credit, Head of Household filing status, and day care credit. This "splitting of the child" is not available to parents who lived together at any time during the last 6 months of the year; then only one of you can claim the child for any tax reasons. The tax benefits may not be split in any other manner.
Note in particular that the non-custodial parent can never claim the Earned Income Credit, Head of Household filing status or the day care credit, based on that child, even when the custodial parent has released the dependency to him.
So, it's good idea to let the other parent know that you will be claiming those items, as many first time divorced parents are not aware of this rule and may try to claim those items, which will cause the IRS to send out letters.
Ref: https://www.irs.gov/publications/p17#en_US_2017_publink1000170897
Scroll down to "Children of divorced or separated parents (or parents who live apart)"
**. The child tax credit (CTC) is limited to your tax liability. The CTC is a non-refundable credit and can only reduce your income tax to 0, It can not help you beyond eliminating your tax liability. But, if you have more than $2500 of earned income, some or all of it is usually given back to you thru the "Additional Child tax credit". That is, part of the CTC may be on line 18b of form 1040 (2019 version) instead of line 13a (lines 17 and 12 in 2018). The ACTC is calculated on form 8812 and is basically 15% of your earned income over $2500. The ACTC is a maximum of $1400 per child (not $2000).
