Unfortunately, because you were unable to rollover the entire amount, whatever money kept apart is taxable, regardless of why you did not roll it over. Had you defaulted on the loan, again that amount would have been taxable.
Please see the links below. The second is Deduction Limits for IRA.
There are other penalty exceptions, but nothing against the tax itself.
Exceptions
Deduction limits