I'm a pastor & receive my housing allowance in box 14 of blank W2 Can I deduct my housing exp's to offset my housing allowance by entering that on a Sch C or form 2106?

I receive my housing allowance reported in box 14 of an otherwise blank W2. Can I deduct my housing expenses to offset my housing allowance by entering that on a Sch C or on form 2106? I realize I won't pay income tax on it, but can the self-employment taxes be mitigated/netted by my housing expenses or does the gross amount need to be subject to self-employment tax? thank you!