It's not the work, it's the income. If you withdrew money from an IRA, 401k, or other retirement plan, or if you cashed in stocks or other investments, that income will be taxable and you must file to report it. You would get tax statements from the various brokers or businesses (1099-INT, 1099-DIV, 1099-R, 1099-B, or others), usually by February 15.
