State tax filing

Thank you for your help.  I must be doing something wrong.  While awaiting brokerage 1099s, I completed everything else in TT Premium for federal and NY.  My NY return showed a refund of about $450 due.  When I entered my brokerage info, I could see the NY refund declining and tax due climbing.  When I finished, NY had gone from about $450 refund due to now I owe NY another $310.  I think my entry of brokerage info should not increase my NY state income tax.  I have looked through the questions and answers and traced through all of the forms.  I see no way to exclude my investment income from taxation by NY.  I have been a legal resident of FL since 2017.  I have not lived in NY since 2017.  I work only remotely in NY with only W-2 income from NY sources and I have appropriate NY income tax withholding.  My investment brokerage income has nothing to do with NY and should not be taxed in NY.  Please tell me what I am doing wrong.  Thank you.