No, I wouldn't worry, once Where's My Refund? shows "Refund Sent" and a date, it could take up to 5 days for it to be reflected in your bank account, it all depends on the bank, how and when funds are credited.
