I am filing for NJ as a resident, and NY as a non-resident. Turbo tax automatically added my NJ business income to NY. Should I remove it from new york state allocation?

Yes, you only want to assign income to New York that you earned while working there. New York will first tax all of your income earned, then reduce the tax based on the percentage of income earned in other states.