Since you have been paying yourself resonable compensation and it appears that it is well documented, there is no issue with taking distributions that should raise a red flag with the IRS or State taxing authorities.
As far as timing and amount, that would depend upon your personal and business needs. Things such as cash flow and future business needs should be considered.
A distribution is normally a tax-free reduction of the shareholder's basis in the S Corp. If the distribution(s) exceed your basis, then such excess would be considered a capital gain.
Here is a link to an IRS article that discusses S Corp stock and debt basis:
https://www.irs.gov/businesses/small-businesses-self-employed/s-corporation-stock-and-debt-basis
