@Csenn1988 your employer should only be withholding taxes in NY, which is where you physically work.
You describe the situation correctly: you will file income tax returns in both states, and it's important to complete the NY return first so that the credit can be correctly applied to the CT tax return. Since I imagine that NY taxes are higher than CT taxes, it could work out that nothing is due to CT related to your salary.
