I live in NYC and work in NJ. I don't have a NY W-4 on file with my employer and thus have no NY taxes taken out of my pay. Could I possibly owe $4,000 to NY?

If I don't have a NY W-4 (why is this needed?) is it possible to owe that much to NYS? I don't understand why the difference is so much.