An attempted Roth conversion of an RMD is a distribution from the traditional IRA and a failed conversion. As a failed conversion, it's an excess contribution to the Roth IRA to the extent that it exceeds the amount eligible to be deposited as a regular Roth IRA contribution for the year. The excess contribution is subject to the normal excess contribution penalties and is corrected by means of a return of contribution before the due date of the tax return for the year in which the failed Roth conversion deposit was made or a regular distribution if done after the due date of that tax return.
Recharacterization was never an option to correct a Roth conversion of an RMD since the RMD was a distribution from the traditional IRA not eligible to be returned to the traditional IRA. The only legal corrective action is the one I described in the preceding paragraph, although I don't doubt that people may have done recharacterizations for this, mistakenly thinking that it was a legitimate corrective action.
