Ideally, zero, if you can afford it. Here's how it works:
1. You rollover $10,000 pre-tax 401(k) into a Roth IRA. You have zero withheld. You owe about $3500 income tax on the conversion (state plus federal) and you pay this from other sources of funds.
Or,
2. You rollover $10,000 pre-tax 401(k) into a Roth IRA and have 30% withheld. This means that only $7000 is converted to the Roth IRA and the remaining $3000 (that was withheld) becomes a taxable distribution from the IRA, subject to income tax plus the 10% penalty for early withdrawal if you are under age 59-1/2. The tax on the $3000 distribution is about $1500 against $3000 of withholding, so you would get the other $1500 back as part of your tax refund.
[I have to edit this answer since this a Roth conversion]
2. You rollover $10,000 pre-tax 401(k) into a Roth IRA and have 30% withheld. This means that only $7000 is converted to the Roth IRA and the remaining $3000 (that was withheld) becomes a taxable distribution from the IRA. The $7,000 Roth conversion is taxed (guesstimate of $2400) and the $3000 distribution is also taxed, plus a 10% penalty for early withdrawal if you are under age 59-1/2 (guesstimate $1350). So you pay more tax overall and end up with less money in the Roth account.
[End of edit]
So you really want zero withheld so all the 401(k) amount is rolled over to the Roth. This means you must pay the taxes out of other funds. If the rollover is in January 2021, you must make an estimated tax payment by April 15, 2021, to both the state and IRS. Depending on your other income, the state tax is 8-13% and the federal tax is probably 22-26%.
Because this counts as income, it may raise your tax bracket, and you might want to do the conversion in steps over a few years. Or you might want to do the conversion in steps if you can't afford the income tax all at once, depending on how much is converted.
