I agree that Form 1040 line 5b must be less than the amount on line 5a, so something is messed up. Have you tried deleting and reentering the Form 1099-R?
For a code-G Form 1099-R reporting the rollover of the entire gross amount from a 401(k) to a Roth IRA, box 1 should show the total gross amount, box 5 should show any after-tax basis and box 2a should should show the difference between box 1 and box 5.
