There is a reciprocal agreement between NJ & PA and you can file Form NJ-165 (http://www.state.nj.us/treasury/taxation/pdf/current/nj165.pdf) with your NJ employer to have them stop withholding NJ taxes and hopefully withhold PA taxes instead.
Anyway, you should file a nonresident NJ return, claiming 0 income there, to get a full refund of the NJ taxes withheld erroneously. It's possible you may have an underpayment of PA taxes penalty but try all of the exceptions to see if any of them stop the penalty.