8 ms·

IBM’s Watson Now A Second-Year Med Student

117 points by jmjerlecki 15y ago

ryanlchan 15y ago
Is this the beginning of companies using web 2.0 tech to solve 'big problems'? Watson might be the first step towards using the social graph and big data for things other than selling more widgets. I'm keen to see how medical professionals start using this tool though. Professional pride and the immense amount of inertia present in medicine could hamper adoption. Worse yet, I can only imagine the level of legal liabilities using a system like this could incur. I'd love to see this adopted and think it could radically reduce costs and increase productivity in the medical field, but I'm going to resign to simple cautious optimism.
- nradov 15y ago
  Clinical decision support systems don't really create any additional legal liability. They just present the user with possibilities. The physician is still legally responsible for making a diagnosis, prescribing a course of treatment, and carrying out medical procedures.
- Jd 15y ago
  Doctors virtually all use lightweight iphone or webapps (e.g. Epocrates) to handle drug information, etc. Like the rest of the world, they no longer memorize massive sets of evolving information, they simply walk around with their Iphone/device of choice and look it up. The providers of such software now and in the future will of course employ teams of lawyers to craft appropriate statements like, "use this as a guide only and not as the basis for a diagnosis or prescription," which of course won't prevent doctors from using it as the basis, but it will keep a healthy sense of legal limbo which exists in many other fields. Basically, I think this doesn't remove the doctor as the interface person, but it allows fewer doctors to see more patients and to streamline the process for many medical problems with (presumably) better outcomes on average. I'm usually not keen on the impersonal nature of modern medicine but what works, works.
- yid 15y ago
  There isn't too much "Web 2.0" tech behind building something Watson. Perhaps the use of ontologies and semantic web datasets, but in academic circles at least that's informally referred to "web 3.0" (that, too, is a bit of a misnomer because Cycorp has been building semantic ontologies for a long, long time).
robobenjie 15y ago
When contemplating what jobs were going to be replaced by robots, I never thought of doctors, but it now seems quite plausible, even probable. (Not that doctors are going to be gone completely, just that their jobs will be forever changed the way that CNC has completely changed the job of a machinist)
- c0riander 15y ago
  I wonder if doctors would become more of a service industry -- doing "patient care" (as customer service) instead of "disease care" now. Could be an improvement for the experience of receiving healthcare; definitely interesting to think about!
  - _delirium 15y ago
    The doctors I know tend to think that it's at least 90% that way already (and has been for a while), at least if you're a primary-care doctor seeing patients in an office, rather than a surgeon or some kind of specialist. The typical patient isn't presenting with a rare disease worthy of a medical TV drama, and instead the main problems are patient-interaction ones: getting the right information out of them, picking treatments that the patients will be satisfied with and follow, figuring out what complicating lifestyle factors they might have, building trust with the patient and family members, etc. With the increasing influence of evidence-based medicine, where doctors are supposed to follow experimentally validated rubrics, the diagnosis itself is often semi-mechanical anyway even when done by a human.
    - ippisl 15y ago
      With increasing automation and crowd-sourcing of customer service, and automated interviewing becoming available to employers , and better computer vision(detecting emotions in live video , and it might be possible to detect lying automatically and maybe more precisely than humans), and the fact that larger part of medical support is done by peer groups (and probably better) i think the building blocks to provide the services you mentioned in highly automated fashion will become available.
    - 15y ago
nradov 15y ago
There is a saying in medical diagnosis: "When you hear hoofbeats behind you, don't expect to see a zebra". https://secure.wikimedia.org/wikipedia/en/wiki/Zebra_%28medicine%29 https://secure.wikimedia.org/wikipedia/en/wiki/Zebra_%28medi... Most patients have common problems that are fairly easy for experienced physicians to diagnose. So there is no point to entering a bunch of data into Watson just to get back the same answer you already know. But it could certainly be helpful for those rare edge cases where the physician can't figure out the "zebra", or where the patient hasn't responded as expected to treatment.
- nostromo 15y ago
  I think this is backwards. (After thoroughly testing of course) I'd rather see Watson be the first-tier of diagnosis, backed up by a doctor when Watson isn't so sure of an answer, as opposed to Watson just helping befuddled doctors. This could really bring down medical costs at scale.
  - astrofinch 15y ago
    And also possibly make it harder for doctors to, say, recommend a more expensive procedure because their hospital gets paid more.
    - ginkgo 15y ago
      Actually, this seems like the perfect opportunity for pharma companies to "help" in creating the database. That's actually what they are already doing when pitching new drugs to doctors. Now they only have to convince the system designers and not thousands of practicing doctors.
      - aheilbut 15y ago
        In the end, the power is going to be with the payors, who are plenty savvy themselves, not the pharma companies. Drugs are a relatively small part (10-15%) of the overall cost of healthcare.
      - 15y ago
deleted 15y ago
[deleted]
robflynn 15y ago
This reminds me of some work I did a few years ago in the oil and gas industry. Not nearly as advanced as this, but we essentially read data from sensors in off shore oil rigs and ran it through our system. We had some pretty nifty pattern recognition stuff in place that would learn from previous data in order to become advanced enough to predict imminent sand breaches during drilling. Given significant warning, it could save the drilling companies lots of money. Katrina ended up severely damaging the rig we were using for testing and I'm not sure we ever went anywhere with it after that. We did do some similar work for detecting trace metals in oil sample analysis records in order to recommend maintenance and detect possible future equipment failures. Anyway, interesting stuff here. I'd love to play around with it. edit Sorry about the double post. I think I got it removed. Connection was a bit flaky.
spc476 15y ago
It looks like Watson is shaping up to the 21st century version of Mycin (http://en.wikipedia.org/wiki/Mycin http://en.wikipedia.org/wiki/Mycin).
janesvilleseo 15y ago
I can see Watson being very helpful to Sergey in his pet project - http://www.nytimes.com/2009/03/12/business/12gene.html http://www.nytimes.com/2009/03/12/business/12gene.html
deleted 15y ago
[deleted]
PostOnce 15y ago
It seems it must be a tool in a tool chain, rather than a physician replacement. Tests must be run, and follow-up questions asked. Also, computers take everything literally, they don't say "Well, that's not what your face is telling me." or think the patient is lying, delusional, or misinformed. You can ask the patient on a scale of 1-10 of their certainty, but again, there are other problems. "What are these red dots?" Also seems non-trivial to diagnose. They can be tested, or more questions can be asked of them, but this seems like an awful lot of work to be managed by a computer alone. It's a good, or even great thing to research, but it seems far from implementation, and that's just from an engineering perspective, imagine the legislative hoops that will need to be leapt through!
- sonoffett 15y ago
  I could definitely see it not as a replacement for a physician, but as sort of a safeguard to check if the physician's decision regarding treatment for a particular set of symptoms matches up watson's previously learned data. Watson wouldn't preempt the physician, but rather raise an alarm if something looks out of the ordinary.
- thaumaturgy 15y ago
  > "What are these red dots?" Also seems non-trivial to diagnose. They can be tested, or more questions can be asked of them, but this seems like an awful lot of work to be managed by a computer alone. I don't entirely disagree with you, but I think you might be underestimating the capabilities of smart programmers, e.g., some of the stuff at the beginning of http://daniellefong.com/2010/02/11/how-law-shapes-the-business-landscape-and-a-patent-puzzle/ http://daniellefong.com/2010/02/11/how-law-shapes-the-busine... If algorithms discussed in that blog post were used in one system (even with pictures taken from an Android device possibly), and that system were to communicate results to Watson ... I can't think of a reason why that wouldn't work. Other than, y'know, patents.
CobraKai 15y ago
As a specialist, I have to say: This makes me very glad I'm not a primary care provider. The doctors of the future will be evaluated on bedside manner and interpersonal skills - not on diagnostic ability or knowledge, because they'll all have access to Watson. Which is scary because smooth talking idiots can get into medical school, but get rapidly weeded out in the real world. It scares me that stupidity can hide behind this computing beast. And there will be situations where Watson cannot help (emergencies) where a doctor and his smarts are all he has to go by. If Watson makes a wrong diagnoses and a patient dies, who will be responsible? The doctor? IBM? All it would take is one or two incorrect diagnoses leading to large malpractice suits to really put the medical profession on edge about this. Granted, many incompetent practitioners exist out there making all kinds of dangerous decisions, but this would really stick out in the public eye.
- arohner 15y ago
  You don't really have to get the diagnosis correct, you just have to get the error bars correct, and then disclose all of that. "Watson believes the best treatment is Foo. It has an X% chance of working, with a confidence interval of Y. The second most likely diagnosis is Bar, with an expected loss of Z if the diagnosis is incorrect. Which would you like to do?" Besides, several specialties are already essentially classifiers (radiologists, anesthesiologists). If machines can't beat humans at classifying cancer from lung scans, they're not far behind. By the time Watson is in production, I bet a computer will be replacing radiologists already.
wensing 15y ago
This is cool. When I was at IBM in Austin last week, one of the IBM-ers mentioned that one of the first areas they hoped to use Watson was primary care. Amazing. I think I'd rather go to Watson than most GP's.
recampbell 15y ago
More details: http://www.ibm.com/developerworks/industry/library/ind-watson/index.html http://www.ibm.com/developerworks/industry/library/ind-watso...
sungam 15y ago
As a physician I doubt this will have much of an impact on the cost of health care. The limiting factor here is not technological but cultural. In the majority of patient encounters a diagnosis has either already been made or is obvious. In principle workers could be trained in the basic clinical skills required in a short period of time and several supervised by a single doctor. There are several reasons that this does not happen to a great extent: (i) Patient's prefer to be assessed by a fully qualified doctor (ii) Medicolegal - who is responsible if a mistake occurs? (iii) Trainee doctors on a per hour basis are often cheaper than employing other workers to perform the same tasks. (iv) For whatever reason Doctor's tend to see patients faster and make decisions more rapidly than other staff e.g. nurses performing the same tasks.
kds 15y ago
"Answering the question “What’s wrong with this patient given these set of symptoms and this family history?” turns out to be a natural fit for the computer." Well, wake me up when the natural-fit question becomes "What’s wrong with this planet given these set of eco-symptoms and this social history?" I'd really like to see what the answer might be. A Post Scriptum Edit/Addition Below ;) The IBM Watson system that recently won on Jeopardy is thoroughly probabilistic and statistical --Norvig (see news.ycombinator.com/item?id=2591154) In a future far, far away when humanity will have gained experience with millions of planets, stars, and human colonies and space-castaway societies - maybe then - IBM Watson will be able to answer my question... Submerging in cryogenic sleep... see you later... <yawning>
JulianMorrison 15y ago
Put it on a public website.

117 points by jmjerlecki 15y ago

Is this the beginning of companies using web 2.0 tech to solve 'big problems'? Watson might be the first step towards using the social graph and big data for things other than selling more widgets. I'm keen to see how medical professionals start using this tool though. Professional pride and the immense amount of inertia present in medicine could hamper adoption. Worse yet, I can only imagine the level of legal liabilities using a system like this could incur. I'd love to see this adopted and think it could radically reduce costs and increase productivity in the medical field, but I'm going to resign to simple cautious optimism.

nradov 15y ago
Clinical decision support systems don't really create any additional legal liability. They just present the user with possibilities. The physician is still legally responsible for making a diagnosis, prescribing a course of treatment, and carrying out medical procedures.
Jd 15y ago
Doctors virtually all use lightweight iphone or webapps (e.g. Epocrates) to handle drug information, etc. Like the rest of the world, they no longer memorize massive sets of evolving information, they simply walk around with their Iphone/device of choice and look it up. The providers of such software now and in the future will of course employ teams of lawyers to craft appropriate statements like, "use this as a guide only and not as the basis for a diagnosis or prescription," which of course won't prevent doctors from using it as the basis, but it will keep a healthy sense of legal limbo which exists in many other fields. Basically, I think this doesn't remove the doctor as the interface person, but it allows fewer doctors to see more patients and to streamline the process for many medical problems with (presumably) better outcomes on average. I'm usually not keen on the impersonal nature of modern medicine but what works, works.
yid 15y ago
There isn't too much "Web 2.0" tech behind building something Watson. Perhaps the use of ontologies and semantic web datasets, but in academic circles at least that's informally referred to "web 3.0" (that, too, is a bit of a misnomer because Cycorp has been building semantic ontologies for a long, long time).

robobenjie 15y ago

When contemplating what jobs were going to be replaced by robots, I never thought of doctors, but it now seems quite plausible, even probable. (Not that doctors are going to be gone completely, just that their jobs will be forever changed the way that CNC has completely changed the job of a machinist)

c0riander 15y ago
I wonder if doctors would become more of a service industry -- doing "patient care" (as customer service) instead of "disease care" now. Could be an improvement for the experience of receiving healthcare; definitely interesting to think about!
- _delirium 15y ago
  The doctors I know tend to think that it's at least 90% that way already (and has been for a while), at least if you're a primary-care doctor seeing patients in an office, rather than a surgeon or some kind of specialist. The typical patient isn't presenting with a rare disease worthy of a medical TV drama, and instead the main problems are patient-interaction ones: getting the right information out of them, picking treatments that the patients will be satisfied with and follow, figuring out what complicating lifestyle factors they might have, building trust with the patient and family members, etc. With the increasing influence of evidence-based medicine, where doctors are supposed to follow experimentally validated rubrics, the diagnosis itself is often semi-mechanical anyway even when done by a human.
  - ippisl 15y ago
    With increasing automation and crowd-sourcing of customer service, and automated interviewing becoming available to employers , and better computer vision(detecting emotions in live video , and it might be possible to detect lying automatically and maybe more precisely than humans), and the fact that larger part of medical support is done by peer groups (and probably better) i think the building blocks to provide the services you mentioned in highly automated fashion will become available.
  - 15y ago

nradov 15y ago

There is a saying in medical diagnosis: "When you hear hoofbeats behind you, don't expect to see a zebra". https://secure.wikimedia.org/wikipedia/en/wiki/Zebra_%28medicine%29 https://secure.wikimedia.org/wikipedia/en/wiki/Zebra_%28medi... Most patients have common problems that are fairly easy for experienced physicians to diagnose. So there is no point to entering a bunch of data into Watson just to get back the same answer you already know. But it could certainly be helpful for those rare edge cases where the physician can't figure out the "zebra", or where the patient hasn't responded as expected to treatment.

nostromo 15y ago
I think this is backwards. (After thoroughly testing of course) I'd rather see Watson be the first-tier of diagnosis, backed up by a doctor when Watson isn't so sure of an answer, as opposed to Watson just helping befuddled doctors. This could really bring down medical costs at scale.
- astrofinch 15y ago
  And also possibly make it harder for doctors to, say, recommend a more expensive procedure because their hospital gets paid more.
  - ginkgo 15y ago
    Actually, this seems like the perfect opportunity for pharma companies to "help" in creating the database. That's actually what they are already doing when pitching new drugs to doctors. Now they only have to convince the system designers and not thousands of practicing doctors.
    - aheilbut 15y ago
      In the end, the power is going to be with the payors, who are plenty savvy themselves, not the pharma companies. Drugs are a relatively small part (10-15%) of the overall cost of healthcare.
    - 15y ago

deleted 15y ago

[deleted]

robflynn 15y ago

This reminds me of some work I did a few years ago in the oil and gas industry. Not nearly as advanced as this, but we essentially read data from sensors in off shore oil rigs and ran it through our system. We had some pretty nifty pattern recognition stuff in place that would learn from previous data in order to become advanced enough to predict imminent sand breaches during drilling. Given significant warning, it could save the drilling companies lots of money. Katrina ended up severely damaging the rig we were using for testing and I'm not sure we ever went anywhere with it after that. We did do some similar work for detecting trace metals in oil sample analysis records in order to recommend maintenance and detect possible future equipment failures. Anyway, interesting stuff here. I'd love to play around with it. edit Sorry about the double post. I think I got it removed. Connection was a bit flaky.

spc476 15y ago

It looks like Watson is shaping up to the 21st century version of Mycin (http://en.wikipedia.org/wiki/Mycin http://en.wikipedia.org/wiki/Mycin).

janesvilleseo 15y ago

I can see Watson being very helpful to Sergey in his pet project - http://www.nytimes.com/2009/03/12/business/12gene.html http://www.nytimes.com/2009/03/12/business/12gene.html

deleted 15y ago

[deleted]

PostOnce 15y ago

It seems it must be a tool in a tool chain, rather than a physician replacement. Tests must be run, and follow-up questions asked. Also, computers take everything literally, they don't say "Well, that's not what your face is telling me." or think the patient is lying, delusional, or misinformed. You can ask the patient on a scale of 1-10 of their certainty, but again, there are other problems. "What are these red dots?" Also seems non-trivial to diagnose. They can be tested, or more questions can be asked of them, but this seems like an awful lot of work to be managed by a computer alone. It's a good, or even great thing to research, but it seems far from implementation, and that's just from an engineering perspective, imagine the legislative hoops that will need to be leapt through!

sonoffett 15y ago
I could definitely see it not as a replacement for a physician, but as sort of a safeguard to check if the physician's decision regarding treatment for a particular set of symptoms matches up watson's previously learned data. Watson wouldn't preempt the physician, but rather raise an alarm if something looks out of the ordinary.
thaumaturgy 15y ago
> "What are these red dots?" Also seems non-trivial to diagnose. They can be tested, or more questions can be asked of them, but this seems like an awful lot of work to be managed by a computer alone. I don't entirely disagree with you, but I think you might be underestimating the capabilities of smart programmers, e.g., some of the stuff at the beginning of http://daniellefong.com/2010/02/11/how-law-shapes-the-business-landscape-and-a-patent-puzzle/ http://daniellefong.com/2010/02/11/how-law-shapes-the-busine... If algorithms discussed in that blog post were used in one system (even with pictures taken from an Android device possibly), and that system were to communicate results to Watson ... I can't think of a reason why that wouldn't work. Other than, y'know, patents.

CobraKai 15y ago

As a specialist, I have to say: This makes me very glad I'm not a primary care provider. The doctors of the future will be evaluated on bedside manner and interpersonal skills - not on diagnostic ability or knowledge, because they'll all have access to Watson. Which is scary because smooth talking idiots can get into medical school, but get rapidly weeded out in the real world. It scares me that stupidity can hide behind this computing beast. And there will be situations where Watson cannot help (emergencies) where a doctor and his smarts are all he has to go by. If Watson makes a wrong diagnoses and a patient dies, who will be responsible? The doctor? IBM? All it would take is one or two incorrect diagnoses leading to large malpractice suits to really put the medical profession on edge about this. Granted, many incompetent practitioners exist out there making all kinds of dangerous decisions, but this would really stick out in the public eye.

arohner 15y ago
You don't really have to get the diagnosis correct, you just have to get the error bars correct, and then disclose all of that. "Watson believes the best treatment is Foo. It has an X% chance of working, with a confidence interval of Y. The second most likely diagnosis is Bar, with an expected loss of Z if the diagnosis is incorrect. Which would you like to do?" Besides, several specialties are already essentially classifiers (radiologists, anesthesiologists). If machines can't beat humans at classifying cancer from lung scans, they're not far behind. By the time Watson is in production, I bet a computer will be replacing radiologists already.

wensing 15y ago

This is cool. When I was at IBM in Austin last week, one of the IBM-ers mentioned that one of the first areas they hoped to use Watson was primary care. Amazing. I think I'd rather go to Watson than most GP's.

recampbell 15y ago

More details: http://www.ibm.com/developerworks/industry/library/ind-watson/index.html http://www.ibm.com/developerworks/industry/library/ind-watso...

sungam 15y ago

As a physician I doubt this will have much of an impact on the cost of health care. The limiting factor here is not technological but cultural. In the majority of patient encounters a diagnosis has either already been made or is obvious. In principle workers could be trained in the basic clinical skills required in a short period of time and several supervised by a single doctor. There are several reasons that this does not happen to a great extent: (i) Patient's prefer to be assessed by a fully qualified doctor (ii) Medicolegal - who is responsible if a mistake occurs? (iii) Trainee doctors on a per hour basis are often cheaper than employing other workers to perform the same tasks. (iv) For whatever reason Doctor's tend to see patients faster and make decisions more rapidly than other staff e.g. nurses performing the same tasks.

kds 15y ago

"Answering the question “What’s wrong with this patient given these set of symptoms and this family history?” turns out to be a natural fit for the computer." Well, wake me up when the natural-fit question becomes "What’s wrong with this planet given these set of eco-symptoms and this social history?" I'd really like to see what the answer might be. A Post Scriptum Edit/Addition Below ;) The IBM Watson system that recently won on Jeopardy is thoroughly probabilistic and statistical --Norvig (see news.ycombinator.com/item?id=2591154) In a future far, far away when humanity will have gained experience with millions of planets, stars, and human colonies and space-castaway societies - maybe then - IBM Watson will be able to answer my question... Submerging in cryogenic sleep... see you later... <yawning>

JulianMorrison 15y ago

Put it on a public website.