The 20% is just an estimate for withholding. The 1099R added to all your other income and may push you into a higher tax bracket. And there is the 10% Early Withdrawal Penalty. You can lose like 50% or more for federal and state taxes and penalties. And by adding more income it can decrease some credits you were getting. It can change many things on your return.
