In our divorce agreement, we agreed to each claim one child as a dependent but both children live with me 80% of the time. Can he legally claim one of the children?

The IRS allows the exemption to the custodial parent (the one with whom the child stayed for the most nights during the year), regardless of what your divorce agreement says. The noncustodial parent can claim the exemption if the custodial parent relinquishes the exemption by signing  Form 8332 or a similar document releasing your claim to the exemption.

If you go against your divorce agreement, the IRS would allow you to claim the exemptions, but there may be repercussions from a legal standpoint that we can't address here.