The first thing you want to do is see what your actual tax liability is. The W4 estimator is a tool you can use.
W4 Calculator
You do want to fill Step 2 of W4 . There is a worksheet on page 3. If you just have two jobs between the two of you, you just check the box on 2C. Remember when you fill your W4 your employer does not see the whole tax picture. So you want to make sure Step 2 is filled either with work sheet or checkmark.