Yes - If you already filed your NY state taxes and stated that you are an NYC resident when you are not, the NY return will include a tax for NYC. So your refund would actually be lower than if you are not an NYC resident.
The chances of the NY Department of Revenue catching this are slim to none.
If you want to see how much more refund you would get as a non-NYC resident, then you will have to prepare an amended NY state income tax return.
Click - How
to Amend a Return You Already Filed
Amended returns must
be printed and mailed, and take up to 12 weeks to process.