Does working as a live-in provider affect social security benefits?

If you're retired and receiving social security as your only income, would working as a live-in caregiver through IHSS affect benefits?  For example, taxed as self-employment.  Presume a tax return also wouldn't need to be filed, but appreciate if you could confirm that as well.