If you rolled over to an IRA an amount that was not eligible for rollover, say, an RMD or a distribution of excess contribution from a 401(k), that constitutes a new, regular contribution to the IRA and any portion of that that exceeds the amount that you are eligible to contribution as a new, regular contribution for the year would be subject to a 6% excess contribution penalty each year the excess remains unresolved.
