Home Sale - adding points to basis

I sold my primary home in 2023.  I paid loan points at initial purchase, and at each of the 4 refinances I did over 26 years prior to sale.  I never deducted any of these points.  Can I now add all of them to the basis for determining gain, or can only add in the ones I paid on the original loan?