Yes, the IRS does charge interest and a late-payment penalty on any balance due under an Installment Payment Agreement. The interest rate is determined quarterly and is the federal short-term rate plus 3 percent. Interest compounds daily.
From the IRS website:
A monthly payment plan or installment agreement gives a taxpayer more time to pay. Though interest still applies, the late-payment penalty is cut in half for any month an installment agreement is in effect. This reduced rate of 0.25 percent (1/4 of 1 percent) per month is only available if the tax return was filed on time.
A user fee will also be charged if the installment agreement is approved. The fee, normally $105, is reduced to $52, if taxpayers agree to make their monthly payments electronically through electronic funds withdrawal. The fee is $43 for eligible low-and-moderate-income taxpayers