Yes. the earnings will be taxed.
If you for example has a basis (after tax money) in the 401K of for example $10,000. That basis will stay the same (unless added to) 10 years later.
So when rolled to a Roth only the basis ($10K in this case) is excluded from being taxed.
