Do I have to fill out the "Two-Earners/Multiple Jobs Worksheet" to change my marital status on my W-4?

You don't have to fill out any part of that form's worksheet ... all you need to fill in is the form itself on the bottom of the first page and hand it in to the employer.