9 ms·
IBM Watson correctly diagnoses a form of leukemia
- apathy 10y agoI can't get past the adblocker spam, but I'm going to guess this woman had APL (acute promyelocytic leukemia, caused almost exclusively by a very specific chromosomal translocation) and the doctors didn't think to look for Auer rods and thus did not try FISHing for the translocation. APL is deadly (5% survival) if treated incorrectly but curable (95%) if treated correctly. However, something or other about that differential has been a standard question on the hematology medical boards for, what, 3 decades now? John Welch, Rick Wilson, Tim Ley and colleagues showed how to do this several years ago: http://jama.jamanetwork.com/article.aspx?articleid=897152 http://jama.jamanetwork.com/article.aspx?articleid=897152 I'm curious whether this case was confounded by cytogenetic complexity, which was part of the problem in that case. Ley later diagnosed Lukas Wartman, a fellow in his lab (or, more like, they worked together to figure out what to do), too: http://oncology.wustl.edu/people/faculty/Wartman/Wartman_Bio.html http://oncology.wustl.edu/people/faculty/Wartman/Wartman_Bio... Lukas did not look nearly as healthy as he does in that picture when I saw him last. He is a great scientist and what he had (adult acute lymphoblastic leukemia, ALL) is a nasty malignancy to treat. Children with ALL tend to do well, but their disease typically seems to arise from different underlying causes than adults, who do poorly. (Not-so-ninja edit: Here is Lukas' own writeup of his experience. He was in a much tighter spot than I recalled: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/ Even children in second relapse are viewed as bad news. Adults in second relapse are generally considered dead men walking. Anyways, read the paper if you're curious.) If Watson caught that the woman had (say) Ph+-like ALL and suggested the right TKI, that will impress me quite a bit, because I've seen someone die from a wrong guess on the latter. On the other hand, if it was APL, they fucked up the differential somehow and are probably kicking themselves. (Double not-so-ninja edit: It wasn't APL-vs-AML, or ALL subtyping. Another poster kindly pointed out the primary source for this news, and something doesn't add up.shrug)
- jacquesm 10y agoIt would definitely be impressive, but it doesn't mean much without knowing how many times it mis-diagnosed. IBM is very good at PR and Watson is milked for every little bit so I'm always a bit suspicious about it in triumphant announcements like these. After all, winning the lottery is a lot less impressive if you've bought a few million tickets and this article says absolutely nothing about any kind of controls or whether or not this was a one-off performance or if it would work at scale.
- apathy 10y agoWell, to play devil's advocate, if all the tickets are free, and the attending gets to ignore all the ones that don't win, that could be useful. FWIW, here's Lukas' writeup of his experience. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/
- smallnamespace 10y ago> if all the tickets are free They're definitely not free; doctors have to evaluate what Watson says, and if Watson is wrong a significant fraction of the time, then doctors will spend a lot of time and energy barking up the wrong tree.
- jfoutz 10y agoEh, it's a tool. Some are useful, some less so. I imagine this would be a tier two kind of thing. It's an unusual case, what suggestions does the robot have? Maybe something obscure and easy to forget.
- apathy 10y agoDoctors have to evaluate what patients say, too, and they're forgetful or clueless at least as often as anyone else. If the cost-benefit analysis favors having Watson (or whatever) around, great, if not, great, just another decision to make.
- 10y ago
- oh_wow 10y agoIBM's Watson scares me primarily because the source and data is not in the public domain. Something this powerful, and this rare should be given to humanity to better it. That said, I'm unsure how doing something like that would provide the right incentives for the research needed to create the next breakthrough.
- deleted 10y ago[deleted]
- codingdave 10y agoI'd say that is exactly why this matters... more automation would let the doctors do more research into new treatments, and let computers run the day to day algorithms (Symptom A gets Test B, followed by results C, which is labelled as diagnosis D, and treatment E.) Having had years of struggle to get a diagnosis on my own problems, the most frustrating part of it all is that exact process, where you are just plugged into the machinery of our health care system, and run through to get a canned answer that needed no human judgment at all, just canned knowledge and a prescribed treatment plan. So just my own opinion, but yes, please let us automate that portion of medicine, so the doctors can do more research and improve things.
- drpgq 10y agoAs a research scientist at a face recognition company, Watson doesn't strike me as particularly powerful or rare. As mentioned in one of the other comments, IBM PR is pretty impressive though.
- cscurmudgeon 10y agoA good example of Dunning Kruger syndrome. -- someone in NLP
- whenwillitstop 10y agoWatson is hyped by the media.
- etangent 10y ago
- airplane 10y agoSomewhat interesting and related is that software that performs medical diagnoses better than doctors has existed since the 1970s (Mycin, an expert system). It seems like nowadays that this kind of solution can possibly be provided at scale at a consumer level (can process new medical knowledge by itself, be installed on commodity hardware), not an expert myself though so I don't know if totally true. What exciting times though. https://en.wikipedia.org/wiki/Mycin https://en.wikipedia.org/wiki/Mycin "it proposed an acceptable therapy in about 69% of cases, which was better than the performance of infectious disease experts who were judged using the same criteria."
- rdtsc 10y agoIt is funny that so many are quick to dismiss this because it is IBM, so "it must be just PR" (saw a few comments here). If Google did this or Elon Musk, everyone would be singing praises and musing about a better, brighter future. Which is funny because it is Google / Facebook / Microsoft which collect and hold everyone's personal details and data. For better or for worse, IBM never seemed to have gotten into that business (yet?) of recording and selling every American's data for ads. In general, I think healthcare is more important than self-driving Teslas or robots making us dinner and scratching our backs. So at least they seem to have focused on the right thing.
- usaphp 10y ago> "I think healthcare is more important than self driving Teslas..." Well self driving cars in theory should save a lot of lives too.
- AsyncAwait 10y agoNot only save, as a disabled person who cannot drive a car, self-driving cars would instantly improve my life experience.
- Zigurd 10y agoYou would see many of the same comments no matter who claimed some kind of first in medical diagnosis. Expert systems have been coming up with diagnoses doctors missed for decades. It's a fat target for automation because much of the expertise is available in diagnostic protocols and few humans can hold all that information in their heads and keep it up to date. MYCIN only used a few hundred rules and it could keep track of which antibiotics to use better than MDs.
- xbmcuser 10y agoYeah I do think the large data sets and processing power available to systems like Watson and Deepmind etc can and should improve detection of many diseases. More than 100k people die in the us alone due to hospital mistakes. 12million misdiagnosed a year is huge number
- dheerthan 10y agoWell, IBM Watson will be surprised to find this little girl, Brittany Wenger built a " Global Neural Network Cloud Service for Breast Cancer " from her bedroom computer around 2012. http://www.qreoo.com/v/dheerthan/316 http://www.qreoo.com/v/dheerthan/316
- seesomesense 10y agoWithout more information, it is difficult to know if this is just another Theranos.
- TillE 10y agoIt's not difficult to see how a computer - especially with a huge database - could do very well at differential diagnosis. It would require the correct inputs, but its advantages are very clear even with fairly simple algorithms. Theranos hand-waved several huge problems with their approach by claiming proprietary magic. What they tried to do is probably impossible. So, big difference.
- vonnik 10y agoOne correct diagnosis is simply not that significant, except for the woman concerned. The real news will be A/B testing on much larger data contrasting Watson's diagnoses with those of doctors sans Watson, and checking both against a ground-truth value, if that is possible. Newsworthy would be a marked bump in accuracy across many cases with Watson...
- YeGoblynQueenne 10y ago>> The technology is certainly there for the eventual creation of an AI version of House "The technology" has "certainly been there" since at least 1970, with software like MYCIN: https://en.wikipedia.org/wiki/Mycin https://en.wikipedia.org/wiki/Mycin MYCIN was an early expert system that used artificial intelligence to identify bacteria causing severe infections, such as bacteremia and meningitis, and to recommend antibiotics It and its ilk (expert systems, with hand-crafted rules) (it was the '70s) where commonly shown to outperform experts: MYCIN was never actually used in practice but research indicated that it proposed an acceptable therapy in about 69% of cases, which was better than the performance of infectious disease experts who were judged using the same criteria. Yet, we still don't have (flying cars) AI that can help doctors make better diagnoses- and note I'm by no means advocating replacing the experts with AI. That would open a whole other can of worms (what do you train your AI on when there's no more experts, because you replaced them all with AI?). But- just having the tech doesn't auto-solve your problems as if by magic. You gotta beat dumb politics first.
- peter303 10y agoEach generation of computer power enables new AI techniques. Expert systems require explicit distillation of "rules". That took a lot of human work. And they are considered brittle when they are outside their expertise. But expert systems could on the million times weaker computers of the 1980s. Watson employes statistical searching of large knowledge bases. You dont have to explicity ferret out all the rules and relationships. Google Translate does this too. There is no preprogrammed language dictionary. The next frontier is deep learning which requires powerful computing to operate in real time. All these techniques have their limitations. No magic bullet yet.
- yread 10y agoIronically, the original story http://www.ndtv.com/health/artificial-intelligence-used-to-detect-rare-leukemia-type-in-japan-1440789 http://www.ndtv.com/health/artificial-intelligence-used-to-d... was also "written" by AI > (This story has not been edited by NDTV staff and is auto-generated from a syndicated feed.)
- soufron 10y agoWow awesome... AI diagnosing diseases have been going around for at least... 30 years? 40 years? Simple decision trees, some data, maybe a bit of neural network, and hop, you've got it. Still, nobody gives a shit about it. You know why? Because the problem is not to make the right diagnosis - it takes only a few seconds to a doctor. It's to set the patient in the right mindset, and to have access to the right treatment.
- yuvalkarmi 10y agoNow that is f-ing cool.
- epmaybe 10y agoI have a slightly unrelated question, but there may be medically trained individuals in this thread. I'm a medical student, and it seems like most people in the field are rather cavalier when it comes to talking about the job outlook for physicians in most any specialty. Do you all think that some healthcare jobs will not be as vital in the next decade thanks to improvements in computing and AI?
- chae 10y agoAlso a medical student with an interest in AI. Healthcare jobs that rely on visual recognition (dermatology, radiology, some pathology) are probably the most likely to benefit in the short term (see Enlitic). Presumably a lot of other jobs require advances in Natural Language Processing/Understanding, as one of the big problems in health is the mostly unstructured nature of the data. It is also possible that many healthcare jobs are essentially AI-complete problems - in this scenario, subjective opinion is not really a reliable marker, but lots of AI specialists give around a 90% chance of human-level machine intelligence by 2070 (there's a table in Nick Bostrom's Superintelligence with the actual figures).
- viach 10y agoThis is a clickbait. Thats it.
- rgoebel 10y agoI see two important parts to this dialog now: 1) machine support for sifting through mountains of data, and 2) the increasing jeopardy of large corporations controlling data. For part 1, the dialog confirms what some of us already know: "Watson" is a branding exercise, and there is no overall coherent plan for strategic AI. But general open source components for literature-based discovery are being built. Part 2 is scarier, and as noted by two speakers from NSF last Friday (Peter Arzberger, Chaitan Baru), "data IS infrastructure," and so the increasing difficulty for public data sources to compete with Google, Yahoo, Facebook, Baidu, etc. is perhaps the more serious challenge for AI support of humans in the future. University research faculty can't tackle problems that require the de novo or nearly de novo creation of such data infrastructure, so we await public policy of funding agencies to clarify the construction and access to such. (Americans in this forum please note that you need to stop writing and thinking as if there is no other part of the planet ... I would be most grateful if Google brokered access to only American data ;-)