See IRS Publication 936 at https://www.irs.gov/pub/irs-pdf/p936.pdf
Page 8 first column
Mortgage ending early.If you spread your deduction for points over the life of the mortgage, you can deduct any remaining balance in the year the mortgage ends. However, if you re-finance the mortgage with the same lender, you can't deduct any remaining balance of spread points. Instead, deduct the remaining balance over the term of the new loan.A mortgage may end early due to a prepayment, refinancing, foreclosure, or similar event.
