If you owe a penalty it will be for the whole year and will be figured on your tax return. Don't worry about it now. Just pay as soon as you can or increase the next one.
I just paid it online. Wouldn't there be any interest for the time period that it was late?
No there will not be interest. The payments you are making are estimated quarterly tax payments. It is only when the total amount you paid for 2019 is calculated that there could be a penalty if you underpaid significantly during the year.