Did you receive a 2018 Form 1099-R for any of this? It seems that you should not have.
I assume that you mean that bank issued the check on 12/31/*2018*. The term "direct rollover" only applies to transactions involving an account in a qualified retirement plan, not to transactions involving IRA accounts on both ends of the transaction. Hopefully the bank actually means that they will code internally it as a trustee-to-trustee transfer where they were both the originating trustee and the receiving trustee, making it neither a distribution nor a rollover and therefore nonreportable. That seems reasonable since the check was not made out to you personally and you therefore never had control of the funds. If the bank treats it as a trustee-to-trustee transfer back to them they should not report it on either a 2018 Form 1099-R or a 2019 Form 5498. Even if the check had been deposited successfully to a Roth IRA for your benefit at Vanguard, this would still have been a nonreportable trustee-to-trustee transfer.
But now I'm a bit confused. Did the bank issue a new check in March or send the original check back to you to resubmit the transfer to Vanguard? Given mention of the bank reclaiming the funds, I'm guessing that the bank reissued the check, perhaps made out differently according to Vanguard's requirements for accepting a trustee-to-trustee transfer.
Had the check been made out to you personally then it would have been a distribution that would normally be subject to a 60-day rollover deadline and the distribution and its rollover would be reportable.
In my opinion, bank personnel are often rather inept when it comes to dealing with IRAs, so you really need to know more about how things are supposed to work than they do so that you can intervene appropriately when they go off track. That's one reason I've studied so much of the IRS documentation and the tax code with regard to retirement account. I also always recommend initiating the transfer of an IRA at the receiving trustee so that the receiving trustee can provide the preferred way of designating the payee of the check or wire transfer. That usually leaves less room for the original trustee to mess things up, but it appears that things got messed up anyway since the funds did not end up in a Roth IRA. A check for a trustee-to-trustee transfer of a Roth IRA to a Roth IRA at Vanguard would most properly be made out to something like "Vanguard, custodian, FBO Jenny Lee Roth IRA," effectively making it payable to your new Roth IRA account. That leaves no doubt that the funds are intended to end up in a Roth IRA and the originating trustee can have some confidence that it will constitute a proper trustee-to-trustee transfer if the originating account was also a Roth IRA. It also puts the blame on the receiving custodian if the funds end up in an account that is not a Roth IRA, perhaps allowing the receiving custodian to make a correction as a correction to a bookkeeping error.
(With regard to IRAs, the terms "trustee" and "custodian" are interchangeable.)