Where do you spend at least 50% of your time?  Where is your driver's license, vehicle registration, voter registration, etc?  Is the NJ home a summer residence?  These may be helpful:

Brief Overview and Filing Requirements (pa.gov)

Part-Year Residents and Nonresidents (state.nj.us)

NJ Division of Taxation - Income Tax - PA/NJ Reciprocal Income Tax Agreement (state.nj.us)

If you work in Philadelphia, there are additional considerations. 

To elaborate on @SweetieJean 's answer, you are dealing with the concept of domicile.  A taxpayer may have more than one residence, but can have only one domicile.  Your domicile is your main, permanent home.  If you are subjected to a residency audit, the state will look at the address you use for your:

• federal tax returns
• insurance records (homeowner’s, car, life)
• passport
• credit cards
• Social Security
• bank and brokerage accounts
• membership organizations

They will also look at the state of your:

- drivers license

- voter registration 

- vehicle registration

- the state you claim as homestead for property tax purposes.

Since NJ and PA have tax reciprocity, you will want your employer to withhold state income tax for your state of domicile.  Reciprocity applies to W2 earnings and means that, if you live in one of those two states and work in the other, you only pay state income tax to your state of domicile.