You have 4 allowances between the 2 of you ... do not double that by both claiming 4.
Rule of thumb ... depending on the income of course ... if the wages are about the same then you each claim 2 but one of you needs to be on the Single rate. If the wages are very different then the higher wage earner can claim them all and married rate but the lower wage earner must use Single and no allowances.
Try these tools:
http://www.irs.gov/Individuals/IRS-Withholding-Calculator
https://turbotax.intuit.com/tax-tools
http://turbotax.intuit.com/tax-tools/calculators/taxcaster
