What is "it" you withdrew money from? A 401(k) or some other retirement account? I also question if you actually withdrew anything. I sounds to me more like you took out a loan against your retirement account, which means you withdrew nothing from the retirement account. It's just the collateral to secure the loan in case you don't pay it back per the terms of the loan.