I assume that each year you have entered your son's name and Social Security number.
If anyone else (including spouse on a separate return) uses the same names and SS numbers, the IRS will sooner or later discover this, and communicate with both of you to find out which parent has the right to take the child as a dependent (it sounds like you do).
If you have not received any such communication, then don't worry about it.
Note that no one at the IRS will tell you who the other person is, if any.
**Say "Thanks" by clicking the thumb icon in a post
**Mark the post that answers your question by clicking on "Mark as Best Answer"