If you were an independent contractor, being paid in cash, etc. or if you received a 1099NEC, as far as the IRS is concerned you were self-employed and need to pay self-employment tax for Social Security and Medicare, as well as ordinary income tax.
You have not said what kind of work you did-----provide some details. Why is the IRS saying you were self-employed?
https://ttlc.intuit.com/questions/2584365-am-i-an-employee-or-an-independent-contractor