Only if you also work in the state your employer is located. What matters is where you earned the income—not where your employer is located.
Example: You're a Vermont resident working in Michigan, where your employer is located. You would file a nonresident Michigan return in addition to your resident Vermont return. However, if you're working remotely in Vermont for a Michigan employer, you'd only file a resident return for Vermont, which is where you earned the money.
That's not to say payroll departments always handle withholdings correctly. Out of habit, or perhaps due to not being aware of how withholding works in these situations, remote employers sometimes withhold taxes for their state, not yours.
In these cases, you will have to file a nonresident return to recover the erroneous withholdings. On the nonresident return, you'd declare earnings of 0 so you can get a full refund for the taxes your employer withheld. You would also want to inform your payroll department of the error to avoid the same problem next year.
Also, due to COVID-19, some states have made special rules and exceptions for teleworkers.