No. You don't have to pay them. They are optional and the IRS isn't expecting them. Unless you will end up owing a lot on your return next April. But usually it's the other way around. If you only pay the last quarters you might owe a penalty for not paying the early quarters when you actually earned the income. But if you paid enough so far it's ok.
