If the IRS owes you money, they don't care if you never file. The late penalty is only for those who owe money to the IRS.
