The process to claim exemption from withholding is described on page 2 of the W-4 form:
"Exemption from withholding. You may claim exemption from withholding for 2020 if you meet both of the following conditions: you had no federal income tax liability in 2019 and you expect to have no federal income tax liability in 2020. You had no federal income tax liability in 2019 if (1) your total tax on line 16 on your 2019 Form 1040 or 1040-SR is zero (or less than the sum of lines 18a, 18b, and 18c), or (2) you were not required to file a return because your income was below the filing threshold for your correct filing status. If you claim exemption, you will have no income tax withheld from your paycheck and may owe taxes and penalties when you file your 2020 tax return. To claim exemption from withholding, certify that you meet both of the conditions above by writing “Exempt” on Form W-4 in the space below Step 4(c). Then, complete Steps 1(a), 1(b), and 5. Do not complete any other steps. You will need to submit a new Form W-4 by February 16, 2021."
Notice that you have to claim that you will have no tax liability.
