NEVER claim exempt ... instead simply add a few more exemptions to line 5 of the W-4. And as to being OK that depends on your annual tax situation ... if this is not OK for the entire year and you forget to change it back then this can have some ugly results later.
Try
these tools to see what the proper withholding should be:
http://www.irs.gov/Individuals/IRS-Withholding-Calculator
https://turbotax.intuit.com/tax-tools
http://turbotax.intuit.com/tax-tools/calculators/taxcaster
