First, you must be a "first time" home buyer. In this case, "first time" means that, during the 2 years prior to the purchase in December 2020, you did not own or co-own any home that you lived in. (You could have rented a home, or even owned a property that was rented to others. But you can't have owned the home you lived in at any time between December 2018 and December 2020.)
Then, you must use the money to buy, build, or "reconstruct" your home, and you must pay for the purchase or reconstruction within 120 days of withdrawing the money. The actual tax code says "reconstruct" although this is often stated as rebuild or renovate. If your home requires substantial rebuilding, it sounds like this would qualify, but beware the 120 day rule. You may have to make several smaller withdrawals rather than one large one, depending on when you need to pay your contractors and suppliers.
Then, you can each use the first time rule if you are both first time homeowners under the rule. The first $10,000 you each withdraw would be exempt from the 10% penalty, but you will still pay regular income tax. If you withdraw more than $10,000, the rest of the money will be subject to income tax plus the penalty. And it's not $20,000 total, it's $10,000 from each account. You will claim the exemption on your tax return but don't send proof of the construction costs -- keep that with your other important papers for as long as you own the home, and only send copies to the IRS if they request more information.
Finally, be aware that this only applies to IRAs and not 401(k)s or other qualified workplace plans. Some people mistake one for the other since they all have the same basic purpose, but they are controlled by different laws and some of the rules are very different. If you have money in an account that is not an IRA, you may be able to roll it over into an IRA and then take your withdrawal.
