The best thing to do is to get a new blank W-4 with both front and back, and carefully complete all the worksheets on both sides with both your and your spouse's expected income and deductions.  This will give you a total number of allowances that you will need to divide between you.  For example, if the worksheet says to claim 5 allowances, then you can claim 5 and your spouse zero, or you claim 2 and your spouse claim 3, etc.