If the father does not file a return he is not a tax payer so you can claim the child as a qualifying relative. But that is ONLY if you can show that you provided more than half of his support. Realize that much of his support is housing provided by the father. See the worksheet below. As mentioned, if he gives you a signed form 8332 it would be much clearer that you qualify to claim him.
https://apps.irs.gov/app/vita/content/globalmedia/teacher/worksheet_for_determining_support_4012.pdf
