If you as a non-resident of NY never actually (physically) worked within New York State, not even for a single day, then your income from that work is not taxable by New York State. As @MayaD stated, NY-source income includes earnings from work performed in New York State, and income from real property located in the state.
If NY taxes were mistakenly withheld from your pay, then you should file a non-resident NY return. On that return show the NY withholdings but allocate zero income to NY. That should result in a full refund to you of the incorrect NY withholdings.
**Answers are correct to the best of my ability but do not constitute tax or legal advice.
