3 ms·

I think all this work could be useful if only those people understood that the technology is not mature enough to remove people from the loop. For the case of

I think all this work could be useful if only those people understood that the technology is not mature enough to remove people from the loop. For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human. This could save a ton of time and help reduce cases where radiologists missed obvious things. My son once broke his arm. I brought him to ER, they made the photo but the two people who looked at the photo said there is nothing wrong with the arm. I asked for the copy of the photo. A week later the swelling did not subside so I took the photo to another doctor and he pointed out an obvious fracture line. There are many ways to deploy automation and I wonder why everybody tries to shoot for removing humans altogether when most of the time this is literally asking for problems.

teaearlgraycold 4y ago
Remove the human and you can charge close to what it costs to employ the human. Aid the human and you’re looking at more like tens to hundreds of dollars per month.
- melagonster 4y ago
  doctor have so many money can spend to save human life. their time expensive than 100$/month.
s0rce 4y ago
I've definitely seen radiologists miss things (as a patient and a researcher) but I've also seen the behavior where if something is labeled or indicated the next person (or in this case the first person) might not give it a true 2nd look and could easily default to agreeing with the AI instead of actually checking things. Overall I think this would be a net benefit as image analysis can help with a lot of these activities, just needs care to not inadvertently remove the humans you still need.
- deleted 4y ago
  [deleted]
- bluGill 4y ago
  Right. Until we trust the AI 100% (at which point we get rid of humans) the AI input shouldn't be given until after the human makes a call. However then the AI gets above 80% (or something like this, UX has better numbers) trusted humans know the AI will look will just skip looking themselves. The above is a common problem that UX research is interested in. I'm not sure how much it is solved, but it goes well beyond medical fields.
  - mistrial9 4y ago
    sadly, this will never appear completely in practice, I predict. Refer to information theory roughly, to support that. That is, pure signal is not often occurring in real systems. Some systems will favor signal quality at the expense of time and fewer false positives and negatives, but the majority of cases in the real world will favor less-effort, less-time decisions that are expedient, or worse, favor obfuscation of the process to cover the sins of the humans benefiting/profiting directly from the show. Also, systems that just are not actually working well, will be sold for quick profit using pressure marketing and forced contracts, at least I think so...
    - deleted 4y ago
      [deleted]
thaumasiotes 4y ago
> For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human. There was a paper a while ago about an effort at something like this. They got the AI going and noticed that the first thing it did was classify all the x-rays by race of the patient. Then they freaked out, gave up on their original project, and wrote their paper about how AI is inherently evil.
kspacewalk2 4y ago
>For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human. The human will still have to look at the x-ray to see if the AI missed something. 95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Maybe it can be used to speed up obvious diagnoses, but it cannot be used to filter and rule anything out. The amount of time a radiologist will spend looking at the x-ray will probably not be reduced, so I don't think there's money to be saved here. A useful productivity tool could be to examine datasets after the radiologist found nothing, as a way to double-check their reading. This won't reduce costs but might marginally improve patient outcomes. Radiologists in first-world medical systems don't really miss a lot of stuff, though. And of course for simple obvious non-life-affecting stuff like broken bones and dental x-rays, you don't need radiologists now either. Your son's x-ray was probably not looked at by a radiologist.
- deleted 4y ago
  [deleted]
- bumby 4y ago
  >95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Isn't this very context dependent? E.g., a delay in lung cancer diagnosis may be a very big deal, but much less so for something like prostate cancer
  - kspacewalk2 4y ago
    Certainly. In fact, prostate cancer is often best left undiagnosed. As the CDC says on its page for prostate cancer screening[0]: >Screening finds prostate cancer in some men who would never have had symptoms from their cancer in their lifetime. Treatment of men who would not have had symptoms or died from prostate cancer can cause them to have complications from treatment, but not benefit from treatment. This is called overdiagnosis. >Prostate cancer is diagnosed with a prostate biopsy. A biopsy is when a small piece of tissue is removed from the prostate and looked at under a microscope to see if there are cancer cells. Older men are more likely to have a complication after a prostate biopsy. [0] https://www.cdc.gov/cancer/prostate/basic_info/benefits-harms.htm https://www.cdc.gov/cancer/prostate/basic_info/benefits-harm...

teaearlgraycold 4y ago

Remove the human and you can charge close to what it costs to employ the human. Aid the human and you’re looking at more like tens to hundreds of dollars per month.

melagonster 4y ago
doctor have so many money can spend to save human life. their time expensive than 100$/month.

s0rce 4y ago

I've definitely seen radiologists miss things (as a patient and a researcher) but I've also seen the behavior where if something is labeled or indicated the next person (or in this case the first person) might not give it a true 2nd look and could easily default to agreeing with the AI instead of actually checking things. Overall I think this would be a net benefit as image analysis can help with a lot of these activities, just needs care to not inadvertently remove the humans you still need.

deleted 4y ago
[deleted]
bluGill 4y ago
Right. Until we trust the AI 100% (at which point we get rid of humans) the AI input shouldn't be given until after the human makes a call. However then the AI gets above 80% (or something like this, UX has better numbers) trusted humans know the AI will look will just skip looking themselves. The above is a common problem that UX research is interested in. I'm not sure how much it is solved, but it goes well beyond medical fields.
- mistrial9 4y ago
  sadly, this will never appear completely in practice, I predict. Refer to information theory roughly, to support that. That is, pure signal is not often occurring in real systems. Some systems will favor signal quality at the expense of time and fewer false positives and negatives, but the majority of cases in the real world will favor less-effort, less-time decisions that are expedient, or worse, favor obfuscation of the process to cover the sins of the humans benefiting/profiting directly from the show. Also, systems that just are not actually working well, will be sold for quick profit using pressure marketing and forced contracts, at least I think so...
  - deleted 4y ago
    [deleted]

thaumasiotes 4y ago

> For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human. There was a paper a while ago about an effort at something like this. They got the AI going and noticed that the first thing it did was classify all the x-rays by race of the patient. Then they freaked out, gave up on their original project, and wrote their paper about how AI is inherently evil.

kspacewalk2 4y ago

>For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human. The human will still have to look at the x-ray to see if the AI missed something. 95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Maybe it can be used to speed up obvious diagnoses, but it cannot be used to filter and rule anything out. The amount of time a radiologist will spend looking at the x-ray will probably not be reduced, so I don't think there's money to be saved here. A useful productivity tool could be to examine datasets after the radiologist found nothing, as a way to double-check their reading. This won't reduce costs but might marginally improve patient outcomes. Radiologists in first-world medical systems don't really miss a lot of stuff, though. And of course for simple obvious non-life-affecting stuff like broken bones and dental x-rays, you don't need radiologists now either. Your son's x-ray was probably not looked at by a radiologist.

deleted 4y ago
[deleted]
bumby 4y ago
>95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Isn't this very context dependent? E.g., a delay in lung cancer diagnosis may be a very big deal, but much less so for something like prostate cancer
- kspacewalk2 4y ago
  Certainly. In fact, prostate cancer is often best left undiagnosed. As the CDC says on its page for prostate cancer screening[0]: >Screening finds prostate cancer in some men who would never have had symptoms from their cancer in their lifetime. Treatment of men who would not have had symptoms or died from prostate cancer can cause them to have complications from treatment, but not benefit from treatment. This is called overdiagnosis. >Prostate cancer is diagnosed with a prostate biopsy. A biopsy is when a small piece of tissue is removed from the prostate and looked at under a microscope to see if there are cancer cells. Older men are more likely to have a complication after a prostate biopsy. [0] https://www.cdc.gov/cancer/prostate/basic_info/benefits-harms.htm https://www.cdc.gov/cancer/prostate/basic_info/benefits-harm...