There are 2 kinds of penalties associated with filing ... late filing & late payment ... so which penalty are they applying to you ?
When did you pay the balance due? If you owed more than $1000 then they can also access an underpayment penalty + interest ... so check the notice carefully. Read up on this at the IRS here :
https://www.irs.gov/taxtopics/tc653
