If my boyfriend and I take turns each year claiming our children, how do I fill out my W4?

The IRS does not care how you fill out the W4.  You can change it every year or leave it the same.  Your goal is to have enough taken out so that you don't owe when you file.  

That's perfect. Thanks George!