"Do I have to file a PA Return to get a refund and then file a NJ Return to pay them??"
Unfortunately yes, that's exactly what you have to do. And no, there is no "other state credit" to be claimed. That's because you actually owe no taxes to PA on your W-2 income.
You should also file PA Form REV-419 with your employer ASAP, to get them to cease the PA withholding. Here's a link to that form:
Employee's Nonwithholding Application Certificate (REV-419) (pa.gov)
**Answers are correct to the best of my ability but do not constitute tax or legal advice.
