I live in MD and my company is based in VA. I employer only withheld MD state taxes. Do I still need to file a VA state return?

No VA return.  VA and MD have a reciprocal tax agreement.

Reciprocal tax agreements allow residents of one state to work in another state without having income taxes withheld in the state they work in. The income they earn in their work state is taxed based on the the tax rules of the state they reside in.

Be aware that the MD-VA reciprocal agreement applies only to wage & salary (W-2) income.