The IRS will not take your child's refund for child support owed by your spouse. They can take your refund if your spouse owes back taxes, child support, state taxes or federal taxes unless you file the injured spouse form.
Here
is a link to IRS Topic 203 which explains refund offsets.