After you file

NO ... you asked for the advance with the rest on the debit card so the refund is slated to go from the IRS to the third party processing bank  to the debit card.  Just because you did not get the advance doesn't change the path your return will take.