I filed my tax return and set it up for direct withdraw for my federal return on 4/15 but the money has not been taken out of the account

[DoninGA]

Your date of payment stays valid.  It may take the IRS up to 10 days to debit your account during this time of the tax season.

• Call IRS e-file Payment Services 24/7 at 888-353-4537 to inquire about or cancel your payment, but please wait 7 to 10 days after your return was accepted before calling.

[DawnC]

The first page of your tax return PDF is the Filing instructions - that page will show the date, account and amount you have scheduled.   It normally takes the IRS a few days to take the  payment, but it is effective the date you scheduled it, and confirmed by your e-file acceptance receipt.  

See the IRS FAQ page