No you can take it all at once. You will each get one 1099R at the end of the year for the total you took out.
For IRS regulations, a "first time" home buyer is someone who did not own or co-own the home they lived in as their main residence at any time in the 2 year period prior to the closing on the new home.
Also, you must withdraw the money before you close, and you must close within 120 days of withdrawing the money. If you withdraw after you close, or it takes more than 120 to close after withdrawing, it won't qualify for the penalty exemption.
Also See Opus 17 answers on this one
https://ttlc.intuit.com/community/taxes/discussion/ira-withdraw-for-first-time-home-purchase/00/2377...
and See IRS publication 590B for IRA Distributions
https://www.irs.gov/pub/irs-pdf/p590b.pdf
