are you married or not? (I am assuming there are no children)

if no, then Single

if married and living with spouse, then Married

if married and not lviing with spouse, then Married- filing single 

I am married and have one kid. As per the non-resident clause, I am supposed to choose single or married filing separately and would like to know if I can claim deductions under section 4(b) of W4 form for the mentioned amount.. can I claim for married which has a higher deduction amount than single?

yes you can.... and these are not 'deductions'.  they are tax "withholdings" 

Yes, you can enter deductions in Step 4(b) of Form W-4.

I don't know what you mean by "can I claim for married which has a higher deduction amount than single?" You cannot select "Married filing jointly" in Step 1(c). Are you referring to the amount on line 2 of the Step 4(b) Deductions Worksheet? You can use the married filing jointly amount if you want to. That will reduce the amount of deductions that you enter in Step 4(b), which will increase the withholding from your pay.

Before you fill out the W-4, be sure to read Notice 1392, Supplemental Form W-4 Instructions for Nonresident Aliens.