Only $10,000 from your IRA and $10,000 from your spouse's IRA can be used penalty free for a first time home purchase. Not $20,000 from one IRA if you are married.
