Did you have a Net Profit on Schedule C for those years? You need a Net Profit of $400 or more to pay self employment tax. Did you enter your income on Schedule C or just as Other Income? What 1040 lines does your self employment income show up on?
