It depends. If the PA taxes withheld were state taxes (found in Box 17 of the PA state line), then yes, you will file two state returns. Since NJ and PA have a reciprocal agreement, you will file a PA return with $0 income to receive a full refund of PA state taxes withheld, and NJ then taxes all of the income to the state.
However, if the PA taxes are only local taxes, you will only file a NJ resident return. Again, because of the reciprocal agreement, you will only pay state taxes in New Jersey. But you can get a credit for PA local taxes paid on your NJ state return.
Please comment what you see on your W2, and I'll be happy to give additional instruction.
I am in the same situation. I downloaded the PA return which I may not need. once that was installed, the credit for Phila taxes disappeared from my return, greatly reducing my NJ refund? How do I fix this?