For tax purposes, who can claim a child depends on several factors. Even though your youngest son is not your ex-husband's biological child, he might still be considered your child's legal parent if the child was born during the marriage. For tax purposes, biological parents and legal parents are treated the same.
In this case, the right to claim the child first goes to the parent that the children lived with most during the year. Even though you share custody, you would need to have had the children with you more often than your ex-husband. If the children lived with you more often than your ex-husband, you would have the right to claim them and would have to give him permission (by providing a copy of a Form 8332). You could provide him permission to claim your oldest two children and claim your son yourself.
If your ex-husband is not your youngest child's legal parent, your son still has to have lived with you more than six months of the year for you to qualify to claim him as a dependent. If he did, you have the right to claim him and your ex husband would not be able to claim him.