19 ms·

In software development sphere, there is a review process for all code that is written. No code gets into production without an explicit approval of some peer o

In software development sphere, there is a review process for all code that is written. No code gets into production without an explicit approval of some peer or supervisor. When you get diagnosed by a doctor, usually no one checks the diagnosis. I would love if my health was checked by medical professional committee. Human and/or AI.

viraptor 3y ago
That's just not correct on both sides. Code reviews happen sometimes in some environments - it's not a given. On the other side, doctors do run their questions by other doctors if they're not sure about the results. (Again, not enforced) Similar for pathology tests results, but with the added layer of the tests themselves being reviewed too. You'd have to join the social network groups for doctors to see that, but there's a lot of double checking happening both with local peers and wider groups. For the more interesting cases anyway... you're not getting a second review if you come in with runny nose and test positive for RSV. (Edit: and even in more complex cases you may not get it for a lot of reasons)
- borski 3y ago
  Genuine question: how do they not violate HIPAA? Are they just crazy careful to abstract and anonymize any identifiable details?
  - viraptor 3y ago
    I'm not based in the US so maybe someone else knows the details there. But HIPAA qualifies the privacy requirements with "and that identifies the individual or for which there is a reasonable basis to believe it can be used to identify the individual." - normally the questions I see are extremely generic, with the sex/age being mentioned at most.
- vidarh 3y ago
  My last two ENTs did not run their answers by anyone. The first failed to correctly record the details of my issue, to the point of writing a summary that said exactly the opposite. The second failed at applying high-school level logic and came to a conclusion that is logically impossible (I tried to walk him through the failure in his reasoning several times before giving up and seeking another opinion). So there may well be a lot of double checking, but there are also a lot of cases of doctors being confidently wrong and not having their reasoning tested. (But, yeah, we have plenty of software developers confidently checking in untested, unreviewed code too)
- krageon 3y ago
  > if they're not sure The thing about mistakes is that you often don't expect you're making one. To my mind there is still value in having a built-in second opinion. The amount of times I've heard doctors tell female friends that their issue is "hormonal" is frankly offensive. Perhaps if we can create an AI (assuming it also doesn't say women have the modern equivalent of "hysterics" for every problem) they will finally get a real answer.
  - viraptor 3y ago
    Don't get me wrong, the system is not perfect and I'm sure it failed many people. It's just the comparison as stated in the original comment that's made up.
- mlajtos 3y ago
  When there is a review process in place, it is usually followed. Mistakes should be expected in the tiniest commits and there should be processes that catch them. As you suggested, medical community does this ad-hoc and only for interesting/unsure cases and this is the result: > The researchers estimate that 7.4 million misdiagnosis errors are made every year, 2.6 million people receive a harm that could have been prevented, and another 370,000 are permanently disabled or die because of the misdiagnosis. Source: https://edition.cnn.com/2022/12/15/health/hospital-misdiagnoses-study/ https://edition.cnn.com/2022/12/15/health/hospital-misdiagno...
  - viraptor 3y ago
    While that's true, we have to keep in mind: 1. We don't know yet what the misdiagnosis rate would be with a review. (There are reasons it could go up) 2. At least in the area I'm familiar with, on many days there just wouldn't be enough staff to do it - people are already sometimes waiting for hours. 3. Issues raised in the article.
    - mlajtos 3y ago
      Why do you think that the ratio of misdiagnosis would go up if there was a dialogue or a review process? 2. and 3. are interlinked. In my mind, medicine is the number one application for ML. Every doctor should have the best AI at their finger tips. One person cannot digest all the research that is being done; computer can. The benchmarks will go up.
      - viraptor 3y ago
        Potential issues from reviews: First person thinks it's X, second person doesn't, first person changes mind but it really was X. First person thinks it's X, second person finds potential issue Y which becomes more important, but X was actually the biggest issue.

viraptor 3y ago

That's just not correct on both sides. Code reviews happen sometimes in some environments - it's not a given. On the other side, doctors do run their questions by other doctors if they're not sure about the results. (Again, not enforced) Similar for pathology tests results, but with the added layer of the tests themselves being reviewed too. You'd have to join the social network groups for doctors to see that, but there's a lot of double checking happening both with local peers and wider groups. For the more interesting cases anyway... you're not getting a second review if you come in with runny nose and test positive for RSV. (Edit: and even in more complex cases you may not get it for a lot of reasons)

borski 3y ago
Genuine question: how do they not violate HIPAA? Are they just crazy careful to abstract and anonymize any identifiable details?
- viraptor 3y ago
  I'm not based in the US so maybe someone else knows the details there. But HIPAA qualifies the privacy requirements with "and that identifies the individual or for which there is a reasonable basis to believe it can be used to identify the individual." - normally the questions I see are extremely generic, with the sex/age being mentioned at most.
vidarh 3y ago
My last two ENTs did not run their answers by anyone. The first failed to correctly record the details of my issue, to the point of writing a summary that said exactly the opposite. The second failed at applying high-school level logic and came to a conclusion that is logically impossible (I tried to walk him through the failure in his reasoning several times before giving up and seeking another opinion). So there may well be a lot of double checking, but there are also a lot of cases of doctors being confidently wrong and not having their reasoning tested. (But, yeah, we have plenty of software developers confidently checking in untested, unreviewed code too)
krageon 3y ago
> if they're not sure The thing about mistakes is that you often don't expect you're making one. To my mind there is still value in having a built-in second opinion. The amount of times I've heard doctors tell female friends that their issue is "hormonal" is frankly offensive. Perhaps if we can create an AI (assuming it also doesn't say women have the modern equivalent of "hysterics" for every problem) they will finally get a real answer.
- viraptor 3y ago
  Don't get me wrong, the system is not perfect and I'm sure it failed many people. It's just the comparison as stated in the original comment that's made up.
mlajtos 3y ago
When there is a review process in place, it is usually followed. Mistakes should be expected in the tiniest commits and there should be processes that catch them. As you suggested, medical community does this ad-hoc and only for interesting/unsure cases and this is the result: > The researchers estimate that 7.4 million misdiagnosis errors are made every year, 2.6 million people receive a harm that could have been prevented, and another 370,000 are permanently disabled or die because of the misdiagnosis. Source: https://edition.cnn.com/2022/12/15/health/hospital-misdiagnoses-study/ https://edition.cnn.com/2022/12/15/health/hospital-misdiagno...
- viraptor 3y ago
  While that's true, we have to keep in mind: 1. We don't know yet what the misdiagnosis rate would be with a review. (There are reasons it could go up) 2. At least in the area I'm familiar with, on many days there just wouldn't be enough staff to do it - people are already sometimes waiting for hours. 3. Issues raised in the article.
  - mlajtos 3y ago
    Why do you think that the ratio of misdiagnosis would go up if there was a dialogue or a review process? 2. and 3. are interlinked. In my mind, medicine is the number one application for ML. Every doctor should have the best AI at their finger tips. One person cannot digest all the research that is being done; computer can. The benchmarks will go up.
    - viraptor 3y ago
      Potential issues from reviews: First person thinks it's X, second person doesn't, first person changes mind but it really was X. First person thinks it's X, second person finds potential issue Y which becomes more important, but X was actually the biggest issue.