Don't claim exempt on your W-4, that should never be done unless you are legally exempt for some reason (such as, you are a diplomat for a foreign country).
If you fill out the W-4 correctly, it will withhold the correct amount based on your income and expected deductions and credits.
The W-4 is 4 pages, not just the front page you turn in. Page 2 has instructions and page 3 is a worksheet to use when you have two jobs, so you can make the right adjustments to the W-4 to account for having 2 jobs. You will want to calculate the adjustments one time, then fill out new W-4s with both employers at the same time.
https://www.irs.gov/pub/irs-pdf/fw4.pdf
Or you can use the online calculator, looks like you are getting some help with that.
