If your tax refund is subject to an "offset" and you want to appeal it, an appeal would have to be made with the agency that submitted the offset.  According to the IRS

You will receive a notice if an offset occurs. The notice will reflect the original refund amount, your offset amount, the agency receiving the payment, and the address and telephone number of the agency.

You should contact the agency shown on the notice if you believe you do not owe the debt or you are disputing the amount taken from your refund.

For more information see: https://www.irs.gov/uac/tax-refund-withholdings-and-offsets