It's not clear what "my retirement" was. Was it a 401(k)? A pension? Something else?
If you still have time to roll it over to an IRA, and you decide to do that, if you can you should put into the IRA the full amount that was in the retirement plan, including the amount of tax that was withheld. That means you would have to come up with money from somewhere else to make up for the withheld tax. It would all get balanced out on your tax return at the end of the year, so in the end you would not pay any tax or penalty on the withdrawal.
