If you don't change your domicile, and your NY abode is temporary, then you are still an NY non-resident even if you live in NY 184 or more days. If you maintain a permanent abode in NY, then you are considered a resident of both states, even if you don't change your domicile. NY seems to offer an offsetting tax credit that is different than the usual tax credit for non-residents.
See the following:
https://www.tax.ny.gov/pit/file/pit_definitions.htm
https://www.tax.ny.gov/pit/credits/resident_credit.htm
You may want to consult with an experienced NY tax professional. Unfortunately, you have chosen to split your time between two states that take the most aggressive stands toward determining residency.
