It depends on your situation. Just like your employer withholds estimated taxes, the retirement plan company can make payments on your behalf too, but it's less common. Look at box 4 of your Form 1099-R (reports withdrawal), which says "Federal income tax withheld". If there's an amount in that box, then they withheld some taxes for an estimate of what you'll owe on the withdrawal (includes the penalty).
After you enter the form 1099-R for taxes, answer the questions that follow carefully. There're some exceptions to the penalty based on what you spent the money on.
