Married filing separately since 2012 in order to reduce the amount I pay for Income Based Fed Loans. Wife contributed to Roth in 2012 to 2016. She made over the $10,000 limit. What should we do? Can I transfer the $12,000 to a traditional to avoid further penalty? Can you explain my penalties? Is it 6% per year until the money is removed from the Roth? So do I owe $144 in 2012, $288 for 2013, $432 for 2014, $576 for 2015? Hopefully can transfer the $2400 to traditional before the 18th to avoid penalty this year.
You are experiencing one of the negative tax consequences of filing separately to get better IBR terms. You need to really think about whether this is a valid long term strategy. (Something else a lot of people don't know -- if you ultimately qualify for loan forgiveness after 10 or 20 or 25 years depending on the program, the forgiven loan principle is taxable income in the year the loan is forgiven. You could owe tens of thousands of dollars, especially if you have unpaid interest that gets capitalized and increases the loan balance.)
Anyway, when married filing separately, she is not allowed to contribute to a Roth if her income is more than $10,000. The penalty for unqualified contributions is 6% per year for as long as the contributions remain in the account and unqualified.
One way to fix this is to file jointly. If you file jointly, her contribution limit is $5500. Since it seems she made a $2400 contribution this year, you can apply the remaining $3100 of limit to qualify $3100 of the original (older) contributions. Then for 2017 if she contributes zero new dollars, you can apply her $5500 limit to qualify an additional $5500 of prior contributions. And so on. Whatever the balance of unqualified contributions is will be subject to the 6% penalty.
Your only other alternative is to withdraw some or all of the money. You can withdraw all of the non-qualified contributions, plus their earnings, before April 18, and pay no penalty. Unless she made qualified contributions in 2011 or earlier, it sounds like you would have to remove the entire balance from the Roth. The earnings get added to your taxable income (since she was not allowed to use a Roth to get tax-free earnings when filing separately). Whatever you don't withdraw before April 18 will be subject to the 6% penalty this year.
You can't roll the money over into another retirement instrument because it will still be non-qualified money. However, you can certainly invest the money in some kind of investment instrument after you withdraw it from the Roth.
Because she is filing separately, she can contribute up to $5500 to a traditional IRA but only if you are not covered by a retirement plan where you work. Note that this is not a rollover. She withdraws the unqualified contributions from the Roth and separately makes a contribution to a traditional IRA.
If you are covered by a retirement plan at your work, she can't make any IRA contributions -- Roth or traditional. You would have to look for investments that are not part of a tax-advantaged scheme (or file jointly).