How do I figure out what years I filed taxes and what years I did not file taxes?

I am trying to determine in what tax years I failed to file taxes with both the Federal government and the Colorado State government. To the best of my knowledge I think I have to file for all years going back to 2016.