For fees to be withheld, there has to be an intermediary. If the IRS mails you the refund, there is no third party to withhold fees.
