You would file a non resident New Jersey return. 

NJ can tax you as a non-resident on your earnings from the days you physically work in NJ.  So you need to keep track of the number of those days.

@Bsch4477 @TomD8 Thank you both for the quick response! 

Should my company withhold my New Jersey state income tax for every pay check?

I didn't find any official statements from "2020 New Jersey Income Tax Nonresident Return Instructions".

If you are liable to pay NJ taxes you should have tax withheld. 

Q.  Should my company withhold my New Jersey state income tax for every pay check?

A.  No. But, it's not unreasonable for them to do so, in your situation.  Ideally, they should only withhold for the pay periods, you actually work in NJ.  But that may be expecting too much from them.  You might try adjusting your NJ-W4 to the minimum withholding. 

https://www.state.nj.us/treasury/taxation/pdf/current/njw4.pdf

Don't have tax withheld.

If your tax is low enough, withholding is not required ( I forget the threshold).

If it is a too large, make NJ estimated tax payments to avoid any penalties.