You can claim 4, per the worksheet,. You do not want any federal income tax withholding because your tax liability will be zero.
There is no correct way to fill out a W-4. A W-4 is only for estimating withholding. It does not lock you in to what you can claim at tax time.
Check your first pay stub, if there is too much income tax withheld, you can adjust. Or, if you prefer, you can just wait til tax time to get a refund. But filing a child dependent gets you refundable credits from the government.
