You are correct. There is no need to prove or support the calculation to the IRS.
The IRS can figure out whether you should have reported the refund as income. They have all the information that is needed to do the calculation. The purpose of the worksheet is for you (or TurboTax) to determine whether you have to report the refund on your tax return, not to prove it to the IRS.
