I use foreign address and I am a Non resident of New Jersey. I can't e-file New Jersey state tax, do I need to pay the filing fee $44.99? How to file my New Jersey tax?

Are you a nonresident of the US?  When did you leave the US and on what visa?  Thank you!

I am a US citizen and live oversea. I visit the US every year. I rent my New Jersey house out and pay federal tax and state tax in the past. I just realized I need to do NJ nonresident filing, is that correct? If so, Which form I should file? Can I do e-filing? Due date?

It is easier for you to look on page 3 and 5 to determine what return you should file.  Due date is April 15, 2019.

Please see attached link for NJ definition of domicile (page 5) and which form to file (page 3). 

https://www.nj.gov/treasury/taxation/pdf/current/1040i.pdf

If you are unable to e-file a NJ return due to a foreign address, you will need to print and mail your return.  I would use a private delivery service is you need to mail  your return.

Where to Mail Your Return

Use the envelope in your booklet to mail your NJ-1040, related enclosures, payment voucher, and check or money order for any tax due. Send only one return per envelope. On the envelope flap are preprinted address labels. To mail your return properly: 1. Remove all labels from envelope flap along perforations; and 2. Attach only the correct label to the front of the envelope.

Payment Due Label Mail returns with tax due (include payment voucher and check or money order, if applicable) to: State of New Jersey Division of Taxation Revenue Processing Center – Payments PO Box 111 Trenton NJ 08645-0111

Refund Label Mail returns requesting a refund (or with no tax due) to: State of New Jersey Division of Taxation Revenue Processing Center – Refunds PO Box 555 Trenton NJ 08647-0555 Do not staple, paper clip, tape, or use any other fastening device for your return and enclosures.