Deducting mileage for medical visits

Hi, I just want to clarify the mileage deduction for medical visits. One only deducts the mileage going to the doctor, and not the round trip back home. Am I correct?  Thanks