[edited] The reason you have withholding from both states may be because you did not file an exemption form with your employer. If you do not want you employer to withhold NJ state taxes, you need to file a form with your employer.
Under the agreement with Pennsylvania, New Jersey does not require withholding from wages, salaries and other compensation paid to Pennsylvania residents, provided Form 165, Employee's Certificate of Nonresidence in New Jersey, is filed with the employer. The certificate is retained by the employer and one copy is sent to the New Jersey Division of Taxation. If a certificate is not filed, the employer must withhold tax in the same manner as from any other employee.
Since there is a reciprocation agreement between PA and NJ, you will not be able to get a tax credit on your PA return for NJ withholding taxes. You will need to file a nonresident tax return in NJ but report no income so that you can get a tax refund for your NJ withholdings. You will then report all of your income on your PA tax return.
For more information about the agreement between NJ and PA, please refer to this link:
http://www.revenue.pa.gov/Pages/default.aspx#.VrvOM9rTmM8
Here is additional
information about filing in multiple states
https://ttlc.intuit.com/questions/1900021-when-would-i-have-to-file-returns-in-more-than-one-state
