I won't be physically working in New Jersey at all this year, but my company has been withholding New Jersey income taxes since I started my job. I want to understand if I will owe NJ taxes and what I might need to ask my company to adjust.
NJ says this:
New Jersey sourcing rules dictate that income is sourced based on where the service or employment is performed based on a day’s method of allocation. However, during the temporary period of the COVID-19 pandemic, wage income will continue to be sourced as determined by the employer in accordance with the employer’s jurisdiction.
https://www.state.nj.us/treasury/taxation/covid19-payroll.shtml
Since your employer is withholding NJ tax, it appears that your income is being "sourced" to NJ, and therefore is taxable by NJ.
And since you are a resident of NY, that income is also taxable by NY. (Your resident state can tax ALL your income, regardless of its source.)
Therefore for 2020 you'll have to file both a non-resident NJ return and your normal home state NY return.
You'll be able to take a credit on your NY return for the taxes paid to NJ on the income that is taxed by both states, so you won't be double-taxed.
In TurboTax, be sure to complete the non-resident return first, before you do your home state return, to make sure the credit is calculated and applied correctly.