We are building our home, haven't been home owners since '14. Can we w/d 10k from his trad. IRA to help pay for the up front fees for construct loan? Penalties/Taxes?

Looking to find out whether or not we can use our traditional IRA money for partial down payment for the building of our home. We have not been homeowners since 2014. This is our first time building a home. We are retired military, but finding a VA lender that will do a construction loan without down payment is almost impossible. So, we are trying to see what is available to us via our traditional IRAs. I have read so much and there seems to be differing advice on this matter. I'm reading that to be considered "first time" home owners, you cannot have owned a home within the last two years. This will be our first home in 4 years, and the first one we've ever built, and the first house we've ever had in Colorado. We've also read that you can use up to 10k towards the house, but sometimes it says purchase of new home and sometimes the word "purchase" isn't included in the verbage. We want to use the money, but we also want to be lawful.