Property improvements done on the house prior to converting it to rental, should have been included in the cost basis at the time you converted it to rental. If you did not do this, you have a major problem. As I understand things, it appears your state also taxes personal income. If so, then this is a double-whammy. You should stop reading now and seek local professional help yesterday, if not sooner.
Property improvements done while the property was a rental should have been entered as such in the assets/depreciation section with depreciation starting on that improvement on the date the work was completed and it was placed in service. Most property improvements do not qualify to be expensed under the Safe Harbor act, and must be capitalized and depreciated. For those property improvements that do qualify to be expensed, if you expense them, then you do not and can not add those costs to your cost basis. They were already deducted as an expense. You can deduct them again by adding to the cost basis. Again, if you did not capitalize and depreciate assets required to be treated that way, seek local professional help.
Property improvements done after the last renter moved out and before you sold it would be added to the cost basis. But since it was never placed in service as a rental asset, you don't enter it in the assets/depreciation section, meaning you can't report the sale in the SCH E section of the program. The sale would be reported instead in the Sale of Business Property section. (Since you don't qualify for the capital gains tax exclusion.)
A local professional can probably help to reduce the fines, penalties and back taxes that may be imposed at both federal and state level, so you can keep more of any gain you may have realized on the sale.
