Yup, just encountered that same problem. Unchecking B5 in the relevant 1099-R makes the logic of the associated 8606-T work to get the correct answer, but the underlying problem appears to be that box B5 is not to being interpreted correctly by 8606-T .
Instead of interpreting 1099-R/B5 to mean "Check this box if the entire amount on line B4 was converted to a Roth IRA", 8606-T is behaving as if 1099-R/B5 means "Check this box if the entire amount of the IRA was converted to a Roth IRA", at which point the entire basis in the traditional IRA is deducted from the distribution instead of just the correct fractional amount.
[Not surprisingly, 2017 TT works correctly when 1099-R/B5 is checked]
