Question on Quarterly Estimated Tax Payments not being equal and skipping.

I have a normal job that have taxes withheld from my check.  I make quarterly estimated payments on my Option Trading profits that I do throughout the year.  I have never made equal payments as I track my option profits and pay the estimated taxes based on those.  In researching my questions, I saw something that said you have to make a payment every quarter and there lies my question as I did not make an estimated payment for September as my profit for that quarter was negative.  I am also looking to tax harvest which I was not planning that will lower my taxes to a point where the first two quarterly payments more than covers what I would owe so I would not need to make a quarterly payment in January for the last quarter and I would be getting a refund for overpayment.  Does this all work out as expected or by not paying anything the last two quarters and getting a refund, will I have any issues?

 

Here is a theoretical example of what I am talking about.

 

1st quarter profit $20K, paid $5K Estimated Taxes.

2nd quarter profit of $40K paid $10K Estimated Taxes.

3rd quarter loss of $10K so didn't make any Estimated Payment

4th quarter profit of $20K but sold $50K Loss for Net Loss of 30K so didn't make any Estimated Tax

 

Expectation is I make a total profit of $20K and paid estimated taxes for $60K so Expect a refund of about $10K

 

EDIT:  One other aspect of my question is am I required to pay quarterly Estimated Taxes as I see there are penalties if I don't pay a quarter.  I have a fulltime job that my wages are taxed.  The only reason I pay quaterly taxes is because I make money in a personal brokerage account and I don't want to be penalized for underpayment as I tend to make a significant amount.