Yes, but not vice versa.
If you are a resident of NY, your NY return includes a credit for paying taxes for nonresident income in NJ.
Your NJ nonresident return doesn't care what taxes you have paid to another state, as there's no related credit. Just attach the NJ 1040NR to the IT-201.
Are you unable to efile NY for a specific reason? Efile is mandatory unless the reason is really good. Efiled returns create these attachments automatically.