You should have been asked about the points in the year you took out the mortgage loan, as they would be considered itemized deductions in whole or in part in that year. It is likely your itemized deductions were not more than your standard deduction and that is why you wouldn't see schedule A with your tax return.
@earnandenjoy
**Say "Thanks" by clicking the thumb icon in a post
**Mark the post that answers your question by clicking on "Mark as Best Answer"
