I am also searching for an answer to exactly the same question. I did some research and to the best of my understanding IRS advises that a Nonresident Alien should write "single" as the marital status and claim only one allowance in federal w-4 form. However, I am not sure if we can claim more than one allowance on State W-4 form by selecting the status as "married".
I am wondering if a J-1 can be considered for "Allowance For Your Spouse: Does your spouse work?" category. If yes, and the spouse doesn't work (although J-1 receives a stipend but I am not sure if it is considered as work for tax purposes) there is an additional allowance which makes the number of allowances equal to 2 (one for yourself and one for spouse given you don't have a child).
Can anyone please elaborate if it's okay to claim more allowance on State W-4 compared to the federal w-4?