6 ms·

> I spent 10 years in mammography software, and I saw first hand how many outside factors can impact a physician's decision to biopsy or not a given artifact o

> I spent 10 years in mammography software, and I saw first hand how many outside factors can impact a physician's decision to biopsy or not a given artifact on an image. This is slightly tangential, but I'm curious about your perspective on a classic example of medical statistical illiteracy. Whenever surveyed, the strong majority of doctors vastly overestimate the odds of a true positive mammogram (half of them by a factor of 10!!), due to flawed Bayesian thinking and the low base rate of breast cancer.[1] Does your anecdotal experience contradict this data? If not, wouldn't two minutes of stats education (or a system that correctly accounted for base rate) utterly swamp intuition-driven analysis of tiny artifacts? Or is it simply that, through folk wisdom and experience, they implicitly adjust other terms in their mental calculus in order to account for this huge error in estimating one factor? [1] https://blogs.cornell.edu/info2040/2014/11/12/doctors-dont-know-bayes-theorem/#:~:text=The%20probability%20that%20a%20woman,(%E2%80%9Cfalse%20alarm%20rate%E2%80%9D) https://blogs.cornell.edu/info2040/2014/11/12/doctors-dont-k...

mjburgess 5y ago
The doctor diagnosing a patient isn't solving a puzzle of the kind posed here. They are doing, as the previous comment said, interpretation. In practice, much of their thinking is profoundly rational and bayesian. Human (and animal) thinking isn't primarily cognitive, ie., explicit reasoning. It is the application of learned concepts to sensory-motor systems in the right manner. We don't look to dr's to formulate a crossword puzzle when a patient arrives; we look to them to be overly attentive to yellow skin when the patient's family has a history of alcoholism.
- gmadsen 5y ago
  I'm not convinced this just couldn't be a large personal data set into an algorithm. Doctors barely have any data as it is. I think personal bio testing and monitoring is going to be a huge market and medical paradigm shift. would you rather have you heart rate and temp constantly monitored for months , or get it checked once a year by a GP to see if you have hypertension or any negative markers
  - mjburgess 5y ago
    With infinite amounts of perfect information and infinite computing time, all question-answer problems are trivially a hash-table lookup.
- wutbrodo 5y ago
  > In practice, much of their thinking is profoundly rational and bayesian. Right, this was the third option I mentioned; I'm certainly not leaping all the way to the conclusion that one shouldn't listen to a doctor about the best course of action after mammogram results[1]. If their explicit understanding of mammography's false positive rate is so incredibly flawed, there is presumably an implicit counterbalance in the calculus that's built on experience (both their own and their mentors'/institution's), or an _order-of-magnitude_ error would show up in patient outcomes. I'd guess that this and the other instances of critical thinking failure that plague medical culture have their rough edges sanded over time, through decades of what is effectively guess-and-check iterative evolution, combined with institutional transmission of this folk wisdom. Though I disagree that I would call this "profoundly rational", as IMO leaving explicit reasoning tools on the table instead of intentionally combining them with intuition/experience/epistemic humility is optimal. Iterative evolution is not an efficient process, and adding an attempt to explicitly model the functions you're learning can be a powerful tool. It's very difficult for me to imagine that a doctor explicitly internalized the basics of Bayesian reasoning wouldn't make them at least marginally more effective in terms of patient outcome, etc. Medical history is full of blind alleys built by medical hubris like your comment's "doctors know A Deeper Truth in their soul that defies explicit logical articulation". (Though I should note I don't claim to have a pat answer to this problem: one can theorize about improving a specific doctor's effectiveness, but scaling it to the whole culture is another, and can bump into everything from supply problems to downstream cultural impacts with negative consequences) [1] Though with knowledge of flaws in such basic reasoning skills in one subpart of the total calculus, a patient can't rationally escape updating in the direction of checking their reasoning more thoroughly. Medicine is a collaborative endeavor between patient and doctor, both bring flaws in their reasoning to the table, and stronger evidence of a huge flaw in reasoning should lower confidence in the overall decision (though at a much lower magnitude, for the reasons we both describe here). This is the same logic that doctors use to rationally discount patient's opinions when they don't perceive them as coming from, eg, overly emotional reasoning.
p1necone 5y ago
This isn't really surprising to me considering that it's not usually a doctors job to estimate the odds of a true postive - false positives are far less of a problem than false negatives when doing an initial scan for cancer.
- wutbrodo 5y ago
  Biopsies aren't cost- and risk-free, including from a patient health and mental health perspective. On top of that, the decision space isn't simply binary: there are multiple types of biopsy procedure, and understanding (even implicitly) the probabilities in play can guide those decisions. The Cult of the Doctor never ceases to amaze (and frankly disgust) me. I think it's absolutely a noble profession, and that doctors are generally smart and conscientious. Even some of the classic complaints are understandable. Eg, my incredibly kind, humble, and open-minded brother-in-law occasionally betrays some extreme disdain for his patients, a common complaint about medical culture. I continue to be a little put off by it, but I understand that he has to deal with some incredibly stupid things from most patients, and it's easy to see how taxing maintaining the welfare/autonomy balance can be. I try to keep this in mind whenever a doctor has an initial defensive reaction to my attempts to be an engaged and informed patient. But, the lengths to which people bend over backwards to make flimsy excuses (like your comment) for massive errors in reasoning is just astonishing. I can't think of a single other context in which people so desperately clutch their pearls at any suggestion that a professional culture or institution could be improved. Doctors are people, and people have flaws. They go through a training and acculturation process, which may also have flaws. This shouldn't be controversial. The only reason I can think of is that patients are no better at dealing with uncertainty than doctors are; the idea that you're being treated by a deity levitating in absolute epistemic certainty is more comforting than the idea that you're dealing with a smart, conscientious, trained human that nonetheless has human limitations (and that the field of medicine itself has plenty of uncertainty).
neaanopri 5y ago
Although we are tempted to think "automate away doctors jobs and take their salaries", it's more achievable and more valuable to provide existing doctors with better UI/UX which helps them make better decisions. Bayesian probabilities could be displayed, other factors presented, and ML could be designed in an explainable way which helps the doctors. But that's too hard, requires too much talking to people, and too much respect for existing institutions, and not enough disruption! Sigh...
- wutbrodo 5y ago
  Yes, I agree. I was responding narrowly to the parent comment's premise, about full replacement (in theory). It's still useful to think about as a theoretical threshold, even if the path to commercialization is almost definitely through assistance.

mjburgess 5y ago

The doctor diagnosing a patient isn't solving a puzzle of the kind posed here. They are doing, as the previous comment said, interpretation. In practice, much of their thinking is profoundly rational and bayesian. Human (and animal) thinking isn't primarily cognitive, ie., explicit reasoning. It is the application of learned concepts to sensory-motor systems in the right manner. We don't look to dr's to formulate a crossword puzzle when a patient arrives; we look to them to be overly attentive to yellow skin when the patient's family has a history of alcoholism.

gmadsen 5y ago
I'm not convinced this just couldn't be a large personal data set into an algorithm. Doctors barely have any data as it is. I think personal bio testing and monitoring is going to be a huge market and medical paradigm shift. would you rather have you heart rate and temp constantly monitored for months , or get it checked once a year by a GP to see if you have hypertension or any negative markers
- mjburgess 5y ago
  With infinite amounts of perfect information and infinite computing time, all question-answer problems are trivially a hash-table lookup.
wutbrodo 5y ago
> In practice, much of their thinking is profoundly rational and bayesian. Right, this was the third option I mentioned; I'm certainly not leaping all the way to the conclusion that one shouldn't listen to a doctor about the best course of action after mammogram results[1]. If their explicit understanding of mammography's false positive rate is so incredibly flawed, there is presumably an implicit counterbalance in the calculus that's built on experience (both their own and their mentors'/institution's), or an _order-of-magnitude_ error would show up in patient outcomes. I'd guess that this and the other instances of critical thinking failure that plague medical culture have their rough edges sanded over time, through decades of what is effectively guess-and-check iterative evolution, combined with institutional transmission of this folk wisdom. Though I disagree that I would call this "profoundly rational", as IMO leaving explicit reasoning tools on the table instead of intentionally combining them with intuition/experience/epistemic humility is optimal. Iterative evolution is not an efficient process, and adding an attempt to explicitly model the functions you're learning can be a powerful tool. It's very difficult for me to imagine that a doctor explicitly internalized the basics of Bayesian reasoning wouldn't make them at least marginally more effective in terms of patient outcome, etc. Medical history is full of blind alleys built by medical hubris like your comment's "doctors know A Deeper Truth in their soul that defies explicit logical articulation". (Though I should note I don't claim to have a pat answer to this problem: one can theorize about improving a specific doctor's effectiveness, but scaling it to the whole culture is another, and can bump into everything from supply problems to downstream cultural impacts with negative consequences) [1] Though with knowledge of flaws in such basic reasoning skills in one subpart of the total calculus, a patient can't rationally escape updating in the direction of checking their reasoning more thoroughly. Medicine is a collaborative endeavor between patient and doctor, both bring flaws in their reasoning to the table, and stronger evidence of a huge flaw in reasoning should lower confidence in the overall decision (though at a much lower magnitude, for the reasons we both describe here). This is the same logic that doctors use to rationally discount patient's opinions when they don't perceive them as coming from, eg, overly emotional reasoning.

p1necone 5y ago

This isn't really surprising to me considering that it's not usually a doctors job to estimate the odds of a true postive - false positives are far less of a problem than false negatives when doing an initial scan for cancer.

wutbrodo 5y ago
Biopsies aren't cost- and risk-free, including from a patient health and mental health perspective. On top of that, the decision space isn't simply binary: there are multiple types of biopsy procedure, and understanding (even implicitly) the probabilities in play can guide those decisions. The Cult of the Doctor never ceases to amaze (and frankly disgust) me. I think it's absolutely a noble profession, and that doctors are generally smart and conscientious. Even some of the classic complaints are understandable. Eg, my incredibly kind, humble, and open-minded brother-in-law occasionally betrays some extreme disdain for his patients, a common complaint about medical culture. I continue to be a little put off by it, but I understand that he has to deal with some incredibly stupid things from most patients, and it's easy to see how taxing maintaining the welfare/autonomy balance can be. I try to keep this in mind whenever a doctor has an initial defensive reaction to my attempts to be an engaged and informed patient. But, the lengths to which people bend over backwards to make flimsy excuses (like your comment) for massive errors in reasoning is just astonishing. I can't think of a single other context in which people so desperately clutch their pearls at any suggestion that a professional culture or institution could be improved. Doctors are people, and people have flaws. They go through a training and acculturation process, which may also have flaws. This shouldn't be controversial. The only reason I can think of is that patients are no better at dealing with uncertainty than doctors are; the idea that you're being treated by a deity levitating in absolute epistemic certainty is more comforting than the idea that you're dealing with a smart, conscientious, trained human that nonetheless has human limitations (and that the field of medicine itself has plenty of uncertainty).

neaanopri 5y ago

Although we are tempted to think "automate away doctors jobs and take their salaries", it's more achievable and more valuable to provide existing doctors with better UI/UX which helps them make better decisions. Bayesian probabilities could be displayed, other factors presented, and ML could be designed in an explainable way which helps the doctors. But that's too hard, requires too much talking to people, and too much respect for existing institutions, and not enough disruption! Sigh...

wutbrodo 5y ago
Yes, I agree. I was responding narrowly to the parent comment's premise, about full replacement (in theory). It's still useful to think about as a theoretical threshold, even if the path to commercialization is almost definitely through assistance.