Mortgage refinance with cash out to pay off 401k loan

In 2014 I took a loan against my 401k for the down payment. In 2020 I refinanced my mortgage and took only enough cash out to payoff the remaining balance of the 401k loan (which was only used to purchase the house). Is that considered a "housing expense" and allow me to deduct all of the interest of the new mortgage loan?