As long as you have net self-employment income of $400 or more, you'll owe self-employment tax, regardless of whether you're eligible to receive Social Security or Medicare. 

There's no age at which you do not pay into Social Security, either as an employee or as a self-employed individual.

That's just the way the tax laws are written.