No, you can't deduct interest paid to the IRS. According to the IRS, beginning with tax year 1991, you can no longer deduct interest you paid to the United States Treasury as an itemized deduction on your Form 1040, Schedule A. So, it's not just the IRS, it's the US Treasury.
You do, however, have to pay taxes on interest the IRS pays you.