Penalties are determined on a quarterly basis. If you did not have a sufficient amount of taxes withheld or paid in estimated taxes to satisfy the required amount for each tax quarter, you are subject to an underpayment penalty for the quarters where there was underpayment.
It's your responsibility to make sure that your tax withholding and estimated tax payments are sufficient to satisfy each quarter's required payment. If the taxes withheld by your employer are insufficient you can give your employer a Form W-4 filled out appropriately to increase your tax withholding.
