I selected 5/4/2020 for my 2019 tax payment to be deducted from my bank account. It has not been deducted. Should I be worried?

Please wait a few more days. The payment may take a bit longer to process through the banking system.

Expect the payment to be taken anytime from now until 7/15 ... the IRS is not processing balance due returns so this may take some time.  So if the DD info is correct just be patient.