Your question is the answer ... if you are below the filing requirement for VA then you will not have to file a VA return and you will own nothing. So don't have any VA withholding.
