5 ms·

Reminder once again that "accuracy" is irrelevant in the real-world. It's only a benchmark metric of the ML research community little concerned with any real-wo

Reminder once again that "accuracy" is irrelevant in the real-world. It's only a benchmark metric of the ML research community little concerned with any real-world impact. Comparing the "accuracy" of an LLM to 9 med students is just rank pseudoscience. The practice of medicine does not produce cute little cue-card prompts with 4 options. Diagnosis means trading the risk of various interventions against their diagnostic value, including a wait-and-see strategy. It means consulting with the patient on impact, as well as conferring with other specialists. It's of less-than-zero value to have an 80% machine accuracy and 75% clinician accuracy if the impact of the machine's mistakes are high risk, unconcerned with the impact of the intervention on the patient, or provide little therauptic upside. Likewise, on these decision benchmarks *Machine decisions are NEVER iterated* -- this makes any claim to "performance" not even borderline pseudoscience, but imv, malpractice. In the realworld practitioner decisions are iterated: doctors do not make high-risk mistake on every sequential decision, given more diagnostic information. It is highly highly unlikely that an inaccurate high-risk judgement call is compounded on each intervention. Whereas machine decisions are never tested in this sequential manner -- if they were, their benchmark performance would drop off a cliff. The production of non-iterated decision benchmarks which measure accuracy and not risk-adjusted real-world impact constitutes, imv, basic malpractice in the ML community. This should be urgently called out before non-experts think that LLMs can give credible medical advice.

EForEndeavour 2y ago
> Reminder once again that "accuracy" is irrelevant in the real-world. Accuracy and other quantitative performance metrics are imperfect for sure, but how else do you propose testing before real-world deployment? How do you propose scalable and feasible testing of human students? > The practice of medicine does not produce cute little cue-card prompts with 4 options. Ah, but the New England Journal of Medicine's Image Challenges (designed to test the knowledge and diagnostic capabilities of medical professionals) does. > It's of less-than-zero value to have an 80% machine accuracy and 75% clinician accuracy if the impact of the machine's mistakes are high risk, unconcerned with the impact of the intervention on the patient, or provide little therauptic upside But this paper does not study the practice of medicine. It intentionally focuses on performance on one specific, well-known medical imaging diagnostic challenge.
- mjburgess 2y ago
  Define a utility across the outcome distribution, eg., profit/quality-adj-life-years-etc. 95% of ML "research" would reveal itself pretty useless if people did this, since the "80-90%" accuracy we're getting is on the "break-even" part of the profit-curve. We're not getting innovation. It's very rare that frequentist stat modelling on historical data would produce anything, in this sense, suprising -- ie., suprise in the utility domain
cityofdelusion 2y ago
The lack of iterative decision making is definitely a huge problem in studies like this. My own experience with GPTs for debugging software show that iterative performance is honestly atrocious. The machine will go all-in on dead-end solutions, forget the context of the problem, and fail to rule things out that should be obvious. Even very simple tasks like locating a misconfigured IDE setting have just awful performance in real world iterative problem solving. It’s not to say the GPTs are currently useless though, they can provide a starting off point or remind us of things we may have missed. Full iterative diagnosis seems really far off though.

EForEndeavour 2y ago

> Reminder once again that "accuracy" is irrelevant in the real-world. Accuracy and other quantitative performance metrics are imperfect for sure, but how else do you propose testing before real-world deployment? How do you propose scalable and feasible testing of human students? > The practice of medicine does not produce cute little cue-card prompts with 4 options. Ah, but the New England Journal of Medicine's Image Challenges (designed to test the knowledge and diagnostic capabilities of medical professionals) does. > It's of less-than-zero value to have an 80% machine accuracy and 75% clinician accuracy if the impact of the machine's mistakes are high risk, unconcerned with the impact of the intervention on the patient, or provide little therauptic upside But this paper does not study the practice of medicine. It intentionally focuses on performance on one specific, well-known medical imaging diagnostic challenge.

mjburgess 2y ago
Define a utility across the outcome distribution, eg., profit/quality-adj-life-years-etc. 95% of ML "research" would reveal itself pretty useless if people did this, since the "80-90%" accuracy we're getting is on the "break-even" part of the profit-curve. We're not getting innovation. It's very rare that frequentist stat modelling on historical data would produce anything, in this sense, suprising -- ie., suprise in the utility domain

cityofdelusion 2y ago

The lack of iterative decision making is definitely a huge problem in studies like this. My own experience with GPTs for debugging software show that iterative performance is honestly atrocious. The machine will go all-in on dead-end solutions, forget the context of the problem, and fail to rule things out that should be obvious. Even very simple tasks like locating a misconfigured IDE setting have just awful performance in real world iterative problem solving. It’s not to say the GPTs are currently useless though, they can provide a starting off point or remind us of things we may have missed. Full iterative diagnosis seems really far off though.