If married filing separately, do we both claim children as dependents?

My husband and I are married filing separately.  My husband has two children from a previous marriage who live with us for 50% of the year.  He has shared custody with the children with his ex-wife. Should both my husband and I claim the children as dependents on our tax returns? According to the custody agreement, each parent has agreed to claim one child on their tax return.  Also, my AGI is greater than my husbands and I pay a majority of all household expenses.