3 ms·
> Unless my doctor is exactly average Just to get back to this point: what if the vision system of your doctor is below average and you augment her by giving h
by bitL 8y ago
> Unless my doctor is exactly average
Just to get back to this point: what if the vision system of your doctor is below average and you augment her by giving her a statistically better vision system, while allowing her to use the additional sources as she sees fit. Wouldn't be that an improvement? We are talking about vision subsystem here, not the whole "reasoning package" human doctors posses.
- timr 8y agoAgain, check that table. It says a lot: https://stanfordmlgroup.github.io/competitions/mura/ https://stanfordmlgroup.github.io/competitions/mura/ On just about every test set, the model is beaten by radiologists. Even the mean performance is underwhelming.
- bitL 8y agoI was referring mainly to this one (from the same group and it actually surpassed humans on average): https://stanfordmlgroup.github.io/projects/chexnet/ https://stanfordmlgroup.github.io/projects/chexnet/ In their paper they even used "weaker" DenseNet-121 instead of DenseNet-169 for Mura/bones. DenseNet-BC I tried is another refinement of the same approach.
- timr 8y agoThose are some sketchy statistics. The evaluation procedure is questionable (F1 against the other 4 as ground truth? Mean of means?), and the 95% CI overlap pretty substantially. Even if their bootstrap sampling said the difference is significant, I don't believe them. Basically, I see this as "everyone sucks, but the AI maybe sucks a little less than the worst of our radiologists, on average"