Remote work - live in NY for CO employer

Hello, 

I work remotely out of New York State for an employer in Colorado. I'm 100% remote, with the exception of two three day trips, where I worked on site in Colorado. I filled out a NYS withholding form and New York taxes are regularly deducted from my paycheck. 

Do I only need to file state taxes in New York, or do I also need to file in Colorado?