3 ms·

It's hard to take seriously the opinions of someone who knows nothing about AI and is therefore unable to critically evaluate the made about Watson. To go from

It's hard to take seriously the opinions of someone who knows nothing about AI and is therefore unable to critically evaluate the made about Watson. To go from any kind of parsing/understanding of medical articles, to actual diagnoses of patients, is absolutely impossible with today's technology

PeterisP 13y ago
Going from symptoms and test results to a diagnosis, and doing it more accurately than humans is certainly possible with current technology. It's not a particular testament to AI quality - it's about the data quantity and the well-known inabilities of humans to accurately judge weights of evidence; humans are really good for effective judgements about common patterns and corelations, however, people are really really horrible at judgement when a multitude of individually rare possibilities are involved, their biases overwhelm the signal.
crusso 13y ago
I'm not sure what you're talking about. I know something about AI. A couple of years ago when Watson destroyed the best human Jeopardy champions, I immediately saw its near-term applicability in sifting through the enormous space of medical data/research and correlating it with patient information to yield amazingly accurate and insightful diagnoses. This isn't after-the-singularity type science fiction where we're adding assumption upon assumption to several levels beyond what is possible. This is applying a technology (that's already proven at performing similar complex deductive analysis over massive quantities of data) to a new discipline. Any assumptions about what we can do in the new discipline with the current technology are minimal and extremely likely overcomable. Even if Watson isn't quite there, the way is pretty clear now. Linear improvements in the Watson approach will yield results.
- cowsandmilk 13y ago
  I would not be surprised if Watson is getting more value from the patient records than the journal articles. Even if patient records are unstructured, they typically are more likely to have key details of import for making rare subtle diagnoses that are not present in journal articles.
- recuter 13y ago
  I used to think that until I dated a medical student. It is heartbreaking to me and I'd grasp at any reason or bit of information to be more optimistic but honestly, get real... How do you think a Venn diagram for medical and computer science (and/or statistics) students looks like? Unlike with jeopardy, you do not have a huge well organized catalog of information lovingly curated by trivia worshiping nerds. It is a ridicules stumbling block but there you have it. Health records are not normalized and you can't touch them anyway, research is closer to human readable prose and filled with jargon - good luck developing NLP for it if you're "just a programmer". Hell, look at their textbooks and the way they study and are trained, that screams for computerized improvement and is still a tall order.
  - hershel 13y ago
    Actually there are already systems in the field that offer better results than people. I remember reading research about decision support systems doing a better job, and there's isabel healthcare , offering decision support system that works great for rare diseases as far as i know. The way they are implemented is by people entering data in an orderly fashion, both those who built the systems ,and doctors . This somewhat limits the system, both from breadth of research and data entry time at the doctor. But even with those limitations , those systems offer far better diagnoses , especially in the rare cases,than you average and even great doctor.
  - PeterisP 13y ago
    You didn't "have a huge well organized catalog of information lovingly curated", however, as the article states, IBM has been working on curating that information for many years already, and they claim that they got all of the knowledge that's taught to med-students already two years ago. So now they can go for the research papers to ensure that the system knows far more than a doctor could read in a lifetime.
  - crusso 13y ago
    You misunderstand how Watson was created. A bunch of trivia worshiping nerds didn't organize and catalog all of its information in the way you're implying. Watson doesn't need perfectly digested raw materials to work with. Watson is able to parse natural language and pull out possibly important information. It then applies some very state-of-the-art machine learning to derive the most likely contextual meaning of that information. That contextual meaning can be cross-referenced with billions of other pieces of weighted information to derive further insights. I come from a family full of doctors. They tend to be smart people, but as Jeopardy showed, even the best player on earth is no match for a machine trained at that discipline. Once we start adding in genetic information, that will allow machines like Watson to customize diagnoses and treatments based upon your specific genetic profile.
- yetanotherphd 13y ago
  On the parsing/understanding side, the main barrier is that it is only possible to infer very simple relationships. Either collecting facts with some simple schema (like "is a" relationships), or inferring mood/topics like "this article seems to be about cancer". On the machine learning side, there is very little in the way of inference. Most of machine learning is an attempt to solve the statistical problem of predicting some value Y (e.g. a diagnosis) given a feature vector X. To obtain data from medical texts (even with human assistance) and use this to form diagnoses of actual patients, would require solving both these problems. We would have to parse and represent the complex logical relationships that are expressed in academic papers. We would then have to have machine learning methods that can work with complex data (not just feature vectors) and combine inference with statistical learning.
  - hershel 13y ago
    Didn't the jeopardy win hinted that watson has inference capabilities ? And isn't doctors and insurance companies being impressed with results are another hint that watson has those capabilities(or is a very big fraud, something IBM wouldn't riskm) ?
drunkpotato 13y ago
I thought his point was pretty clear, and I am having trouble figuring out how you so completely missed it.
bayesianhorse 13y ago
Diagnosis is actually way easier than understanding medical articles... For that matter it is hard to coordinate and record treatments if the process of diagnosis is too complicated or arcane for AI to understand.

PeterisP 13y ago

Going from symptoms and test results to a diagnosis, and doing it more accurately than humans is certainly possible with current technology. It's not a particular testament to AI quality - it's about the data quantity and the well-known inabilities of humans to accurately judge weights of evidence; humans are really good for effective judgements about common patterns and corelations, however, people are really really horrible at judgement when a multitude of individually rare possibilities are involved, their biases overwhelm the signal.

crusso 13y ago

I'm not sure what you're talking about. I know something about AI. A couple of years ago when Watson destroyed the best human Jeopardy champions, I immediately saw its near-term applicability in sifting through the enormous space of medical data/research and correlating it with patient information to yield amazingly accurate and insightful diagnoses. This isn't after-the-singularity type science fiction where we're adding assumption upon assumption to several levels beyond what is possible. This is applying a technology (that's already proven at performing similar complex deductive analysis over massive quantities of data) to a new discipline. Any assumptions about what we can do in the new discipline with the current technology are minimal and extremely likely overcomable. Even if Watson isn't quite there, the way is pretty clear now. Linear improvements in the Watson approach will yield results.

cowsandmilk 13y ago
I would not be surprised if Watson is getting more value from the patient records than the journal articles. Even if patient records are unstructured, they typically are more likely to have key details of import for making rare subtle diagnoses that are not present in journal articles.
recuter 13y ago
I used to think that until I dated a medical student. It is heartbreaking to me and I'd grasp at any reason or bit of information to be more optimistic but honestly, get real... How do you think a Venn diagram for medical and computer science (and/or statistics) students looks like? Unlike with jeopardy, you do not have a huge well organized catalog of information lovingly curated by trivia worshiping nerds. It is a ridicules stumbling block but there you have it. Health records are not normalized and you can't touch them anyway, research is closer to human readable prose and filled with jargon - good luck developing NLP for it if you're "just a programmer". Hell, look at their textbooks and the way they study and are trained, that screams for computerized improvement and is still a tall order.
- hershel 13y ago
  Actually there are already systems in the field that offer better results than people. I remember reading research about decision support systems doing a better job, and there's isabel healthcare , offering decision support system that works great for rare diseases as far as i know. The way they are implemented is by people entering data in an orderly fashion, both those who built the systems ,and doctors . This somewhat limits the system, both from breadth of research and data entry time at the doctor. But even with those limitations , those systems offer far better diagnoses , especially in the rare cases,than you average and even great doctor.
- PeterisP 13y ago
  You didn't "have a huge well organized catalog of information lovingly curated", however, as the article states, IBM has been working on curating that information for many years already, and they claim that they got all of the knowledge that's taught to med-students already two years ago. So now they can go for the research papers to ensure that the system knows far more than a doctor could read in a lifetime.
- crusso 13y ago
  You misunderstand how Watson was created. A bunch of trivia worshiping nerds didn't organize and catalog all of its information in the way you're implying. Watson doesn't need perfectly digested raw materials to work with. Watson is able to parse natural language and pull out possibly important information. It then applies some very state-of-the-art machine learning to derive the most likely contextual meaning of that information. That contextual meaning can be cross-referenced with billions of other pieces of weighted information to derive further insights. I come from a family full of doctors. They tend to be smart people, but as Jeopardy showed, even the best player on earth is no match for a machine trained at that discipline. Once we start adding in genetic information, that will allow machines like Watson to customize diagnoses and treatments based upon your specific genetic profile.
yetanotherphd 13y ago
On the parsing/understanding side, the main barrier is that it is only possible to infer very simple relationships. Either collecting facts with some simple schema (like "is a" relationships), or inferring mood/topics like "this article seems to be about cancer". On the machine learning side, there is very little in the way of inference. Most of machine learning is an attempt to solve the statistical problem of predicting some value Y (e.g. a diagnosis) given a feature vector X. To obtain data from medical texts (even with human assistance) and use this to form diagnoses of actual patients, would require solving both these problems. We would have to parse and represent the complex logical relationships that are expressed in academic papers. We would then have to have machine learning methods that can work with complex data (not just feature vectors) and combine inference with statistical learning.
- hershel 13y ago
  Didn't the jeopardy win hinted that watson has inference capabilities ? And isn't doctors and insurance companies being impressed with results are another hint that watson has those capabilities(or is a very big fraud, something IBM wouldn't riskm) ?

drunkpotato 13y ago

I thought his point was pretty clear, and I am having trouble figuring out how you so completely missed it.

bayesianhorse 13y ago

Diagnosis is actually way easier than understanding medical articles... For that matter it is hard to coordinate and record treatments if the process of diagnosis is too complicated or arcane for AI to understand.