If your brother takes any amount from a 401k, that amount is taxable. If he chooses residential care, the portion of his expenses paid for medical care is deductible as a medical expense. If the primary reason he lives there is to receive medical care, then all of it would be deductible as a medical expense.
If he has someone come to the house, to provide help with normal household activities like cooking and cleaning,then he could take a Child care Credit as a disabled adult but the maximum expense he could use for the credit is $3,000. (It's just called Child Care, but can also be used for disabled adults if they are unable to care for themselves.) If he has a medical person, such as a nurse, coming to care for him at home, those medical services would be deductible as medical expenses.
Keep in mind that he could only deduct the portion of medical expenses over 7.5% of his income, and the total of all his itemized expenses, including medical, would have to exceed his standard deduction in order to benefit from itemizing.
Your question of how much he would have to pay tax on is both simple and complex. The answer is that he would owe tax on the full amount distributed from his 401k. The amount of tax he would actually pay would depend on how much of his Social Security becomes taxable due to receiving other income.
First, his total income amount is determined by adding together his other income plus 1/2 of his SS benefits.
If his filing status is single, and his total income is between $25,000 and $34,000, up to 50% of his benefits could be taxable. If his combined income is more than $34,000, up to 85% of his benefits could be subject to tax. And, if his combined income is less than $25,000, his benefits are not taxable at all.
So, you would add 1/2 of his benefit amount to different distribution amounts to determine how much of his benefits become taxable. All of his distribution amount is taxable, plus whatever portion of his SS benefits become taxable. From that, he would receive a $12,000 standard deduction in 2018, so any amount of taxable income over that $12,000 is the amount on which he would actually pay tax. If he itemizes and has deductible expenses over $12,000, substitute that amount for the $12,000.
