For the penalty exception, you can't have owned your main home (where you lived) for at least two years before closing on the new home. Each IRA owner can withdraw up to $10K to buy or build a "first time" home and be exempt from the 10% penalty for early withdrawals. You will always owe regular income tax on the withdrawal. If you both own IRAs, you can withdraw $10K each and be exempt from the penalty, but you can't withdraw $20K from one IRA.
