I am recently married but my spouse lives outside the U.S. Should I file as married or single? Should I update my W4 to married?

(The note on my W4 states that I should keep my status as single if my spouse is a non-resident alien.  Is that accurate)?