Y was incorrect in not doing what Z did to correct the reporting. It's not their responsibility to make sure that the funds end up in the correct account. Their reporting (or absence of reporting) should be based only on their own actions. I probably would have pushed Y harder to make the correction.
Unless on your tax return you treated as a rollover the distribution reported by Y, it's likely that the IRS will detect a tax underpayment and send you a bill. If the receiving custodian treated it as a transfer, omitting it from Form 5498, it might be difficult to convince the IRS that this was indeed a trustee-to-trustee transfer. If reported as a rollover, I'm skeptical that the IRA would detect a violation of the one-rollover-per-12-months rule, but reporting it as a rollover might make it difficult to later assert that it was actually a trustee-to-trustee transfer that should not have been reported should the IRS challenge the rollover.
It's for reasons like this that trustee-to-trustee transfers of IRA generally be initiated at the receiving custodian. This allows the receiving custodian to dictate to the original custodian the proper way to specify the payee on the transfer, which should be Custodian FBO John Doe IRA or Custodian FBO John Doe Roth IRA depending on the type, making it clear that it's a transfer.
Banks are notorious for messing up this sort of transaction when initiated at originating bank. Front-office personnel are frequently oblivious to the difference between a distribution and a transfer, and the error is often the result of the front-office person marking the wrong box on a form. The form is then processed by the back office based on the marking of the form. Initiating the transfer with the receiving custodian can avoid dealing with the ill-trained front-office bank personnel in favor of having the transfer processed only by the better-trained back-office personnel.
