No, retirement benefits are taxed according to your state of residence instead of where the retirement funds originated. You would not need to file a non-resident Virginia return based on your retirement benefits coming from a Virginia source.
Take a look at the following information about Virginia non-resident filing requirements:
Virginia Non-resident Filing Requirements