Actually that's a misconception. You don't have to request a refund if you deducted fees from your federal refund because your not actually 'charged' unless you receive it five days early. The summary of charges only apply if that event happens.
