I am under the impression that i can use this money tax free if we have not purchased a house in the last two years.
Not tax free. Up to $10,000 can be distributed without the additional early distribution penalty but the normal tax on the distribution still applies.
See IRS pub 590B for details. (The same rules for a Traditional IRA apply to a SIMPLE IRA for first home).