The expenses for home improvements/remodeling your own residence are not deductible and do not get entered on your tax return. Save all your records for someday when you sell the house.
I assume this is for your primary residence, and not for rental property or any other type of business property. In a nutshell, yes.
If your original loan balance was $75K at the time of refinance, and you refinanced at $100,000 for a $25,000 cash out, then you can only claim 75% of the interest on SCH A for the life of the loan.
If you use every penny of that $25K to improve the property (flooring, kitchen, bath, etc.) then no problem, you can claim 100% of the interest on the loan as a SCH A deduction.
However, if you took the loan out in 2020 and do not use the money to improve the property in 2020, then you can only claim 75% of your interest on SCH A on your 2020 tax return. Then if you use that money to pay for your improvements in 2021, you'll be able to claim 100% of the interest as a SCH A deduction on your 2021 tax return.