Who can claim the child if the child lives with the mother and the father lives somewhere else and has only lived with the child for 3 months

our child was born in December 2016, I claimed him on taxes. his father lived with us for 3 months before we broke up. he says he can get more for the child even though he hasn't filed taxes in 3 years, & the child doesn't live with him, & he only seen him 3 times since we broke up.