How do I avoid Underpayment Penalty when I did a large IRA to Roth IRA conversion in December and paid the estimated taxes right away?

I did the conversion in mid December and paid the estimated taxes the same day but i'm still being hit with a $185 penalty because I apparently didn't pay this across 4 quarters through the year when I didn't know I would do this conversion. This can't be right?