I live and work remotely from NJ. My company is in NYC. I pay taxes to NY instead of NJ. How do I fix it? Since NY won't easily refund me

I think I'm paying taxes to NY by mistake since I never come the office in NY. I know that it's not easy to prove to NY. What do I do?