Stevehofwa, at a minimum I would forever retain in your records copies of all of the Forms 5498 and Forms 1099-R, and any tax returns for the corresponding years. For many aspects of reporting on From 8606 the statute-of-limitations clock does not start to run until Form 8606 is files because From 8606 is considered to be a separate tax return for certain purposes. Also note that section 408(o)(2)(C)(ii) the tax code requires nondeductible traditional IRA contributions to be reported on your tax return (that's one of the purposes of Form 8606). If the IRS ever examines your tax returns the IRS will not recognize your contributions as having added to your basis in nondeductible contributions that would have reduced the taxable amount of your Roth conversions. See Failure to report nondeductible contributions in IRS Pub 590-A.
Unless there was never any investment gain while the funds were in a traditional IRA, there was probably some taxable amount that should have been reported, but small amounts would not have made a significant difference in your tax liability. However, if you have other traditional IRA accounts (including SEP, SIMPLE and rollover IRAs), your past tax returns could have substantially underreported taxable income and tax liability, potentially extending the statute of limitations for the IRS to be able to assess the tax.
