You should check your spam email folder. You should have gotten like 4 emails before they took it out of your account. Now when the IRS sends your refund they won't take the fees out again, except maybe the Extra $40 Refund Processing Service charge (45 in California).
