Keep in mind that if you are under age 59½ and have no other penalty exception, the entire gross amount distributed from the IRA, including the amount withheld for taxes, will be subject to a 10% early-distribution penalty unless you come up with the money to complete a rollover of the gross amount distributed within 60 days of the distribution from the IRA. The fact that you use the money for tax withholding does not itself exempt you from this penalty. To make up for underpayment of taxes early in the year sometimes one will take an IRA distribution and have a majority of it withheld for taxes, then complete the rollover of the distribution using other funds. However, considering that you had a 401(k) loan, coming up with the money to complete the rollover might be a challenge for you.
Note that if the loan came due because you left the company, you likely did not default on the loan but the loan was instead paid off with an offset distribution. You have until the due date of your tax return for the year of the offset distribution, including extensions, to come up with the money and complete a rollover to an IRA of the amount of the offset distribution. Rolling over some or all of an offset distribution would allow continued deferral of taxes on the amount rolled over, also avoiding the early-distribution penalty on the amount rolled over.
