On a second home, points can only be deducted over the life of the loan. The same is true for refinances, except in cases where you used a portion of your refinance proceeds to improve your home. In that case, the points related to the home-improvement portion of the loan can be deducted in the year you paid them.
If you refinanced with the same lender, any undeducted points left over from the first mortgage will be deducted over the life of your new loan. But if you refinanced with a new lender, any leftover undeducted points can be deducted in the year of the refinance.
For more information, refer to the TurboTax Frequently Asked Question "Can I deduct mortgage points?" below:
https://ttlc.intuit.com/replies/3302279