Yes, you should. You will file a Non-Resident tax return for New Jersey, and a Resident tax return for Virginia. Prepare the New Jersey tax return first, then Virginia.
Make sure to adjust out your income in the New Jersey state section to show you didn't earn anything in the state. Also, contact your payroll or human resources department (if you haven't already) to have that changed.
How to file a non-resident state return?
https://ttlc.intuit.com/replies/3302052