Very complicated. See 26 U.S. Code § 7701 - Definitions | U.S. Code | US Law | LII / Legal Information Institute (cornell.... Section 7701(d)(1)(A)(i) and others.
The way I read it is if you keep your "green card" you are a US tax payer. If you have physical investment income (eg. rental properties), you are a US tax payer. But if you have neither and simply invest the monetary instruments in the US (eg. stocks) you may not be a US tax payer.
Your tax situation in you home country will also influence the results as foreign taxes are often part of the calculation.
Bottom-line is it's going to cost you fees from a qualified Tax Attorney to get the correct answer.
