if you take the standard deduction, then the whole issue is moot to you.....
did you do a cash out refi at some part along the way?
if you did a cash out refi and you itemize, then there is an issue. If that is you case, please explain what a) the mortgage balance was when you did the cash out refi (the prior mortgage), b) what the cash out balance was after the refi, what you did with the money (a-b) and c) what the mortgage balance is now. What we need to understand is now much of the cash you took out went to improve the house vs. the rest.