Are W2 workers who live in NYC but work in NJ supposed to get the majority if not all of their NJ state taxes back?

If you were a full-year or part-year resident of New York State and you had income sourced to and taxed by NJ, you may claim a nonrefundable resident credit against your New York State tax. This credit is allowable only for the portion of the tax that applies to income sourced to and taxed by NJ while you were a New York State resident.

For more information see, IT-112-R-I, Instructions for Form IT-112-R New York State Resident Credit.