No penalty if you left that job at age 55 or more.
However, you might be in violation of the rules for the 401 plan itself, so check with the plan's administrator, or read the plan's documents.
