9 ms·

This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product. Right now much of the mammogram reading is extremely strai

by tedivm 5y ago

This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product. Right now much of the mammogram reading is extremely straight forward. Radiologists can fly through those exams with high quality, especially when you consider the macros they have setup that allow them to write most of the report with with a two second voice command. This is one of the biggest things radiologists complain about- from their perspective most AI is solving problems that aren't actually useful to radiologists. I think the companies that will do best are the ones that augment radiologists with a focus on making them more accurate and efficient, rather than trying to cut them out of the loop (I'm a bit bias though as my previous company, Rad AI, is taking the augmentation approach and it has worked out rather well).

kn1ght 5y ago
Hey I've also worked in a previous company that was doing exactly this for DK, UK, etc. As far as I remember one of the bigger things there was tissue density and masking effects where it might become not as clear cut for a radiologist. We saw a lot of opportunity in that area and in prognosis- the whole health economics aspect of frequency of visits.
- jacquesm 5y ago
  > tissue density and masking effects But those are also harder for software based systems.
bob33212 5y ago
Good point. Similar to intellisense while coding.
- 908B64B197 5y ago
  But coding is a creative task. Breast cancer screening is more like classifying breeds of dogs or determining if there's a fire hydrant in the picture.
version_five 5y ago
Just commenting to agree with this. I've done a lot of ML use case ideation work, and what happens so often is the ML folks latch on to something that ML may be able to do, but that doesn't move the needle at all in terms of doing the work faster or better. Often, the ML provides an extra layer that slows things down, even if it's working at state of the art AUROC or whatever we're trying for.
quietbritishjim 5y ago
> This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product. I think you're disagreeing with a point that the parent comment didn't make. It's worthwhile in general to continue AI research. The properties of this application make it ideal for testing and progressing AI technology. But if AI doesn't end up being used in this particular area in the short or medium term, that's another matter entirely.
tw04 5y ago
> This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product. There's a difference between: not ideal to tackle and not ideal to build a business around. It is absolutely ideal to tackle which will hopefully build confidence in a system that can then be expanded to solve harder problems. If you can't even solve this problem I don't see how you're getting public confidence in the harder ones.
- antupis 5y ago
  Yeah this also third world needs this kinda solutions where you can replace specialist with algorithm.
  - earthboundkid 5y ago
    Why? The defining feature of the third world is cheap labor. The third world might need more effective training, so they can convert more cheap labor into specialists, but I don't see why per se they would prefer to use a computer sold at Western prices to a local specialist.
    - Siira 5y ago
      "Cheap effective training;" Pick two.
antasvara 5y ago
Is there an argument to be made that decreasing the workload on radiologists could improve their results in more difficult cases? I don't think it's a stretch to imagine that less tired/busy doctors would be able to perform their job more effectively. There's also the fact that radiologists get paid a pretty decent salary. If I could devise a system that can do a highly paid worker's job faster, cheaper, and at a similar level of effectiveness, why wouldn't I? It also opens up that skilled worker's schedule and allows them to tackle more difficult to interpret scans.
- 908B64B197 5y ago
  > I don't think it's a stretch to imagine that less tired/busy doctors would be able to perform their job more effectively. You are suggesting something that's against the groupthink for that profession. You can't graduate from an MD program while holding these thoughts. I'm being 100% serious here.
  - danachow 5y ago
    > You can't graduate from an MD program while holding these thoughts. By your logic I must not exist then. Like who do you think is labeling your training images? Look at the authors of pretty much all cited studies in that meta-analysis. Sure see a lot of MD and MD-PhD there.
- tedivm 5y ago
  That argument is exactly the one made by my former company, Rad AI. Rather than focusing on trying to "solve" radiology they're focused on augmenting radiologists. Their main goal is to reduce the amount of time a radiologist spends on each report while increasing accuracy and reducing errors. The point I'm making is that focusing on mammograms is not the way to do that. It's already a very optimized specialty. If AI researchers worked more closely with practicing radiologists (not academic ones) I imagine there's all sorts of areas where existing technology today can improve things.
  - antasvara 5y ago
908B64B197 5y ago
> This is one of the biggest things radiologists complain about- from their perspective most AI is solving problems that aren't actually useful to radiologists. "This is one of the biggest things horses complain about- from their perspective most cars are solving problems that aren't actually useful to horses." > I think the companies that will do best are the ones that augment radiologists with a focus on making them more accurate and efficient, rather than trying to cut them out of the loop "I think the companies that will do best are the ones that augment horses with a focus on making them faster and efficient, rather than trying to cut them out of the loop" Seriously, I'm not surprised a profession that built itself a legalized cartel with the AMA would fight tooth and nail something that might actually automate away their job, or flat out use their monopolistic position to collectively decide to boycott a particular software vendor/tech they perceive as threatening.
- scoopertrooper 5y ago
  They can destroy as many textile mills as they like, but the economic incentives of technology will win the day.
  - bjornsing 5y ago
    Probably. But it can take a long time.
- divbzero 5y ago
  Though it’s plausible that ML could eventually take on some screening or diagnostic tasks from highly trained radiologists or pathologists, there is the hurdle of accountability that will need to be tackled. As with self-driving technology, the models will have to be good enough to take on the liability risk at scale.
  - 908B64B197 5y ago
    > the hurdle of accountability The medical field is notorious for delivering "efforts", not results (unlike engineering). No SLA here.
scoopertrooper 5y ago
> Right now much of the mammogram reading is extremely straight forward. Radiologists can fly through those exams with high quality, especially when you consider the macros they have setup that allow them to write most of the report with with a two second voice command. That to me sounds like an ideal automation use case. You have an entire class of highly skilled professionals performing a repetitive task. Sure, they may be able to do it quickly, but I imagine they must have to do a bunch of them each day. You could imagine a world where hospitals didn't have radiologists on staff, but rather they acted in a consulting capacity for difficult cases.
- acituan 5y ago
  > That to me sounds like an ideal automation use case. You have an entire class of highly skilled professionals performing a repetitive task Walking on variable surfaces is also a highly repetitive task and automation is yet to master it.
- throwaway4220 5y ago
  Very few hospitals have radiologists on staff
ska 5y ago
> The fact is there's no market for the product. That's empirically not true; the first such product approved for clinical use entered the market in the late 90s, and they've had small-to-medium success since. Definitely none of them changed things radically, but it's pretty common CAD functionality now and has both detractors and champions. > Radiologists can fly through those exams with high quality, This is also not entirely true, one of the biggest problems for screening (not diagnostic) programs is that the false negative rate for radiologists is high. They do fly through these scans, but that's to make it economically feasible. Also, the average radiologist performance on screening mammo is not very good - most have to do it regularly (probably 100s per month) to do it well. You are right that augmentation is the more plausible path to clinical success, but that's been what most of the clinical systems aim to do anyways, provide a sanity check on the radiologist.
- ethbr0 5y ago
  I have no idea why everyone tries to solve the hardest problem (true positives, no false positives) with machine learning, instead of solving the much easier problem (true negatives, no false negatives) that often delivers similar value. E.g. If a system could tell me which 967 results out of 1000 are normal, and send the other 33 for human review. True positives require solving every problem in the solution space. True negatives only require solving the most frequent problems.
  - ska 5y ago
    If I remember correctly this was tried in Europe ages ago, liability concerns made it impractical in US.
    - hellbannedguy 5y ago
      Why do I feel the AMA was a bigger problem than liability? Plus--wouldn't the software company be liable? (I don't think the tech is ready for office use, but when it is--I still think it will be a battle.) Kinda like all medical devices. Whenever there's a problem, the equipment is looked at first? Personally, I don't think the AMA wants computers taking away their doctor's precious income, and will only acquiesce when the tech makes the doctor more money, or is so good politicians/insurance companies start demanding it.
version_five 5y ago
I'm really surprised by the pushback you're getting. I'd be curious the background of the people that have come back with replies about how it's an ideal use case, etc. This misunderstanding of the problem domain is exactly why these projects always fail - people come up with a "solution" that solves something that isn't the problem, despite whatever clever horse analogies people can come up with.
nl 5y ago
There are a bunch of countries outside the US where access to high quality radiologists is a significant problem. But yes, augmenting human intelligence instead of substituting it often does work better.
deleted 5y ago
[deleted]

by tedivm 5y ago

kn1ght 5y ago

Hey I've also worked in a previous company that was doing exactly this for DK, UK, etc. As far as I remember one of the bigger things there was tissue density and masking effects where it might become not as clear cut for a radiologist. We saw a lot of opportunity in that area and in prognosis- the whole health economics aspect of frequency of visits.

jacquesm 5y ago
> tissue density and masking effects But those are also harder for software based systems.

bob33212 5y ago

Good point. Similar to intellisense while coding.

908B64B197 5y ago
But coding is a creative task. Breast cancer screening is more like classifying breeds of dogs or determining if there's a fire hydrant in the picture.

version_five 5y ago

Just commenting to agree with this. I've done a lot of ML use case ideation work, and what happens so often is the ML folks latch on to something that ML may be able to do, but that doesn't move the needle at all in terms of doing the work faster or better. Often, the ML provides an extra layer that slows things down, even if it's working at state of the art AUROC or whatever we're trying for.

quietbritishjim 5y ago

> This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product. I think you're disagreeing with a point that the parent comment didn't make. It's worthwhile in general to continue AI research. The properties of this application make it ideal for testing and progressing AI technology. But if AI doesn't end up being used in this particular area in the short or medium term, that's another matter entirely.

tw04 5y ago

> This is actually why it isn't ideal for AI to tackle. The fact is there's no market for the product. There's a difference between: not ideal to tackle and not ideal to build a business around. It is absolutely ideal to tackle which will hopefully build confidence in a system that can then be expanded to solve harder problems. If you can't even solve this problem I don't see how you're getting public confidence in the harder ones.

antupis 5y ago
Yeah this also third world needs this kinda solutions where you can replace specialist with algorithm.
- earthboundkid 5y ago
  Why? The defining feature of the third world is cheap labor. The third world might need more effective training, so they can convert more cheap labor into specialists, but I don't see why per se they would prefer to use a computer sold at Western prices to a local specialist.
  - Siira 5y ago
    "Cheap effective training;" Pick two.

antasvara 5y ago

Is there an argument to be made that decreasing the workload on radiologists could improve their results in more difficult cases? I don't think it's a stretch to imagine that less tired/busy doctors would be able to perform their job more effectively. There's also the fact that radiologists get paid a pretty decent salary. If I could devise a system that can do a highly paid worker's job faster, cheaper, and at a similar level of effectiveness, why wouldn't I? It also opens up that skilled worker's schedule and allows them to tackle more difficult to interpret scans.

908B64B197 5y ago
> I don't think it's a stretch to imagine that less tired/busy doctors would be able to perform their job more effectively. You are suggesting something that's against the groupthink for that profession. You can't graduate from an MD program while holding these thoughts. I'm being 100% serious here.
- danachow 5y ago
  > You can't graduate from an MD program while holding these thoughts. By your logic I must not exist then. Like who do you think is labeling your training images? Look at the authors of pretty much all cited studies in that meta-analysis. Sure see a lot of MD and MD-PhD there.
tedivm 5y ago
That argument is exactly the one made by my former company, Rad AI. Rather than focusing on trying to "solve" radiology they're focused on augmenting radiologists. Their main goal is to reduce the amount of time a radiologist spends on each report while increasing accuracy and reducing errors. The point I'm making is that focusing on mammograms is not the way to do that. It's already a very optimized specialty. If AI researchers worked more closely with practicing radiologists (not academic ones) I imagine there's all sorts of areas where existing technology today can improve things.
- antasvara 5y ago

908B64B197 5y ago

> This is one of the biggest things radiologists complain about- from their perspective most AI is solving problems that aren't actually useful to radiologists. "This is one of the biggest things horses complain about- from their perspective most cars are solving problems that aren't actually useful to horses." > I think the companies that will do best are the ones that augment radiologists with a focus on making them more accurate and efficient, rather than trying to cut them out of the loop "I think the companies that will do best are the ones that augment horses with a focus on making them faster and efficient, rather than trying to cut them out of the loop" Seriously, I'm not surprised a profession that built itself a legalized cartel with the AMA would fight tooth and nail something that might actually automate away their job, or flat out use their monopolistic position to collectively decide to boycott a particular software vendor/tech they perceive as threatening.

scoopertrooper 5y ago
They can destroy as many textile mills as they like, but the economic incentives of technology will win the day.
- bjornsing 5y ago
  Probably. But it can take a long time.
divbzero 5y ago
Though it’s plausible that ML could eventually take on some screening or diagnostic tasks from highly trained radiologists or pathologists, there is the hurdle of accountability that will need to be tackled. As with self-driving technology, the models will have to be good enough to take on the liability risk at scale.
- 908B64B197 5y ago
  > the hurdle of accountability The medical field is notorious for delivering "efforts", not results (unlike engineering). No SLA here.

scoopertrooper 5y ago

> Right now much of the mammogram reading is extremely straight forward. Radiologists can fly through those exams with high quality, especially when you consider the macros they have setup that allow them to write most of the report with with a two second voice command. That to me sounds like an ideal automation use case. You have an entire class of highly skilled professionals performing a repetitive task. Sure, they may be able to do it quickly, but I imagine they must have to do a bunch of them each day. You could imagine a world where hospitals didn't have radiologists on staff, but rather they acted in a consulting capacity for difficult cases.

acituan 5y ago
> That to me sounds like an ideal automation use case. You have an entire class of highly skilled professionals performing a repetitive task Walking on variable surfaces is also a highly repetitive task and automation is yet to master it.
throwaway4220 5y ago
Very few hospitals have radiologists on staff

ska 5y ago

> The fact is there's no market for the product. That's empirically not true; the first such product approved for clinical use entered the market in the late 90s, and they've had small-to-medium success since. Definitely none of them changed things radically, but it's pretty common CAD functionality now and has both detractors and champions. > Radiologists can fly through those exams with high quality, This is also not entirely true, one of the biggest problems for screening (not diagnostic) programs is that the false negative rate for radiologists is high. They do fly through these scans, but that's to make it economically feasible. Also, the average radiologist performance on screening mammo is not very good - most have to do it regularly (probably 100s per month) to do it well. You are right that augmentation is the more plausible path to clinical success, but that's been what most of the clinical systems aim to do anyways, provide a sanity check on the radiologist.

ethbr0 5y ago
I have no idea why everyone tries to solve the hardest problem (true positives, no false positives) with machine learning, instead of solving the much easier problem (true negatives, no false negatives) that often delivers similar value. E.g. If a system could tell me which 967 results out of 1000 are normal, and send the other 33 for human review. True positives require solving every problem in the solution space. True negatives only require solving the most frequent problems.
- ska 5y ago
  If I remember correctly this was tried in Europe ages ago, liability concerns made it impractical in US.
  - hellbannedguy 5y ago
    Why do I feel the AMA was a bigger problem than liability? Plus--wouldn't the software company be liable? (I don't think the tech is ready for office use, but when it is--I still think it will be a battle.) Kinda like all medical devices. Whenever there's a problem, the equipment is looked at first? Personally, I don't think the AMA wants computers taking away their doctor's precious income, and will only acquiesce when the tech makes the doctor more money, or is so good politicians/insurance companies start demanding it.

version_five 5y ago

I'm really surprised by the pushback you're getting. I'd be curious the background of the people that have come back with replies about how it's an ideal use case, etc. This misunderstanding of the problem domain is exactly why these projects always fail - people come up with a "solution" that solves something that isn't the problem, despite whatever clever horse analogies people can come up with.

nl 5y ago

There are a bunch of countries outside the US where access to high quality radiologists is a significant problem. But yes, augmenting human intelligence instead of substituting it often does work better.

deleted 5y ago

[deleted]