This is my take on this, not legal advice:
The Form 1099-R reports a direct rollover to a traditional IRA. However, depositing this into a Roth IRA transformed this transaction into an irrevocable taxable rollover to a Roth IRA, appropriately reported on the Form 5498 and an erroneous amount in box 2a of the Form 1099-R. There is no method provided under the tax code to allow undoing the error of rolling the 401(k) distribution over to a Roth IRA instead of to the intended traditional IRA.
Because moving the money from the Roth IRA to a traditional IRA cannot be done by recharacterization after tax year 2017, the distribution from the Roth IRA in 2023 constitutes a regular distribution, subject to Roth IRA ordering rules and potentially to a 10% early-distribution penalty depending on your age and how much basis you have in regular Roth IRA contributions, regardless of how the IRA custodian reports it, unless this distribution from the Roth IRA is rolled over to a Roth IRA. Because a traditional IRA is not an account that is eligible to receive a rollover from a Roth IRA, the deposit of this money into the traditional IRA constitutes a regular contribution for 2023 and is an excess contribution to the extent that it exceeds the amount that you are eligible to contribute for 2023.
It seems to me that the best outcome would accomplished by getting the money back into the Roth IRA. To do this you would have to obtain a return of contribution from the traditional IRA, then deposit an amount equal to the amount distributed from the Roth IRA into the Roth IRA as a rollover of the original Roth IRA distribution (regardless of the gain or loss adjusted amount of the distribution from the traditional IRA). If this rollover is more than 60 days after the date of the distribution from the Roth IRA, the Roth IRA custodian would have to accept a late rollover under IRS Notice 2020-46 with the reason being either deposit into an account that you thought was an eligible account (the traditional IRA) or, more likely, based on financial-institution error (their improper instruction that a correction could be made by recharacterization). Such a rollover is subject to the one-rollover-per-12-months rule, so you must not have done any traditional IRA-to-traditional IRA or Roth IRA-to-Roth IRA rollover in the one-year period preceding the distribution from the Roth IRA.
With the funds back in the Roth IRA, you would owe the tax but no penalty. On the plus side, gains in the Roth IRA will be tax free once your Roth IRAs are qualified. If the amount distributed from the traditional IRA as a return of contribution is more than the amount originally rolled over to the Roth IRA, the difference will be taxable on your 2023 tax return.
