My son only lived with me all of 2020 and my ex only visited my son a total of 32 times in the year of 2020. Can my ex legally claim my son on his taxes?

If your son meets the qualifications as your dependent  and lived with you more than half of the year only you can claim him unless you allow the other parent to do so.