6 ms·

I work for a medical software company which evaluated Watson for use in our diagnostic product. It should be seen as a Good Thing that medical is a Serious Fie

I work for a medical software company which evaluated Watson for use in our diagnostic product. It should be seen as a Good Thing that medical is a Serious Field where Non-Serious Ideas get run through the ringer and fail. And that's what happened with Watson. The word from our engineers was that Watson was designed to sell commercials, not be a useful tool in a problem space like differential diagnosis or drug reactions. I think this is good. Financial has the same reputation. You can't just use agile to bust out a minimum viable medical or financial product. You will be chewed up and spit out. And rightfully so. Because when it comes to our health, or our money and our investments, we don't have tolerance for failure. OK, my social network double posted my communication, annoying but it's just a communication. However "my financial tool just double posted a transaction", or "my prescriptions tool just doubled the dose of a medicine" -- these are not OK states at all. I wish Watson lived up to the hype, but sadly, our experience was that it does not. So it failing out of Healthcare isn't a surprise to us, nor is it a bad thing.

yosyp 8y ago
Tangentially related: how does one get their foot in the door at a medical software company? Is there a big market for such professionals?
- pram 8y ago
  Learn MUMPS and move to Wisconsin.
  - walshemj 8y ago
    Is MUMPS still a thing :-) maybe I ought to dust of my RT-11 / FORTRAN skills off
    - prepend 8y ago
      The biggest EMR company was Epic and uses Mumps [0]. However, they are losing share to Center and others [1], so this formula will only work for a little while longer. [0] https://news.ycombinator.com/item?id=13860937 https://news.ycombinator.com/item?id=13860937 [1] https://ehrintelligence.com/news/cerner-epic-mckesson-among-top-5-global-health-it-vendors https://ehrintelligence.com/news/cerner-epic-mckesson-among-...
      - sergers 8y ago
        Epic still dominates the EHR space. They have deployments slotted for 3 years. If you have ever worked with epic, they offer "managed" solutions, which you either do the epic designed way, or don't go epic... They have enough market share and business to dictate how things will work in a hospital env
- tjr225 8y ago
  I was recruited right out of college to work at Cerner. Its probably still fairly trivial to get a job there, but you have to live in Kansas City.
- dbs 8y ago
  You get at the door of companies in the healthcare space that need to get up to their game and tell them what you can do for them. That's what I did.
phren0logy 8y ago
Oh, I agree if it doesn't work it should be tossed aside. My concern is that given the potential usefulness demonstrated in so many other domains, I suspect that many of the tech failures in healthcare are due to the fact that it's too expensive to take the time to get it right.
- mlthoughts2018 8y ago
  I think this might also be supported by looking at some of the better funded / successfully exited health tech startups. I’m thinking of things like Flatiron Health, BenevolentAI, Quartet and Spreemo, and surely many more. At heart these companies seem to shy away from being “really about” medicine, and try to be a more watered down data company, looking at health records, social media, and other data to provide recommendation, service matching, comorbidity analysis, hospitalist tools, cost tools, assistance program allocation tools. I’ve been very interested in ML applications in health, but a lot of the businesses that seem capable of getting funding seem like they are super light / elementary on the statistical modeling side, and the value add is just a claim to modernize crappy claims data or unify a bunch of previously disparate health data sources. Maybe more advanced use of modeling will come later, though I am skeptical just given the way that once big enterprise customers dig hooks into essentially consulting services for health records, they won’t let go. Another thing that creeps me out is when you see e.g. an ex Palantir board member joining health record data companies. It’s not hard to understand why a big insurance company, or at worst even government agencies, want decision tools on top of huge stores of health record data. Do we really benefit from some super new / unproven startup building up that data set & tooling?
roymurdock 8y ago
Theranos is a counterexample. Probably would have been outed as fraudulent if they tried to enter a more safety-critical space (ventilators, pumps, injectors, etc.), but still a bad sign for medical diagnostic industry that it got so much funding and so far along, inking huge deals before collapsing. Perhaps some good will come if it makes others in the space more cautious in the short run Next industry to keep an eye on (many are already) is automotive. There are a lot of vendors trying to put a lot of software/connectivity into the next generation of cars. Lots of regulation but I will opt for a dumb car over new smart stuff any day. Same with a lot of different traditionally embedded systems that are now being connected to the internet
- abakker 8y ago
  Arguably, the big deals and optimism indicate that there is a lot of pent up demand for successful things, and people are willing to take some risk to get solutions. It’s not all doom and gloom, solutions just have to work.
  - sk5t 8y ago
    Conversely, I read Theranos's progress as evidence of cynical bets that the government/military could likely be directed into pumping billions of USD into the company regardless of its merit.
    - abakker 8y ago
      You could certainly draw that conclusion from its original board. I did as well. Still, I think the point stands that people want healthcare to work, and will pay for it. It just so happens that it takes a lot of effort to improve the system since what we have is pretty good. (Not to say it doesn’t have problems)
- nimish 8y ago
  No, Theranos started to fail as soon as anyone with real medical expertise without a stake in the company started to probe it. Theranos was extremely successful at fleecing VCs and embarrassing political board members. It's a good thing that it got so far, because it proves just how bad VCs are at evaluating successful companies and how much of the funding game is who you know.
megaman22 8y ago
> The word from our engineers was that Watson was designed to sell commercials, not be a useful tool in a problem space This is the number one thing to remember about Watson. I've worked with teams within IBM that spent years and thousands upon thousands of man-hours trying to get it to be useful in a fairly well-defined, constrained use-case. Eventually they bailed and went to what was essentially a simple decision tree. It hits it out of the park on the sexy buzz-words, but I'm ready to curse the whole field for giving customers wildly inflated expectations.
- Spooky23 8y ago
  I worked on a project that did a little Watson work as well. Same result as you described. The shame is decision tree based expert systems can be pretty powerful, but the tools these guys were using were not that great.
  - davak 8y ago
    I am a nerdy physician desperate to work with somebody on decision tree based expert systems. Are there a bunch of existing projects that I’m just missing?
    - gtmb 8y ago
      Well, I just started an open source project to gather the data for such systems. It is starting as a database/graph of symptoms and conditions. But the vision is bigger than that. https://github.com/gafmgafm/med https://github.com/gafmgafm/med There are similar products but they are not open source, such as: http://apimedic.com/ http://apimedic.com/ and http://www.diseasesdatabase.com/ http://www.diseasesdatabase.com/
      - mindcrime 8y ago
        I've been doing a lot of work lately with automated Abductive Inference, and medical diagnosis has been cited as one of the key uses of same. Perhaps there could be some opportunity to collaborate (everything I do is open source as well, btw). If you guys would like to talk, feel free to hit me up at prhodes@fogbeam.com
sergers 8y ago
I had posted this previously in a different thread, but it's all smoke and mirrors. I work in the diagnostic imaging space, we are working on integrating AI for assisted diagnosis or screening. It's a several-year road map for some base function. We had a top sales executive leave for IBM Watson Healthcare division about a year ago, was very proud to be part of what Watson was going to bring to healthcare He's been working back at my company, couldn't stand selling lies (and wasn't making much commission with canceled contracts). From a named company more impressed with what Google deepmind has done with NHS

yosyp 8y ago

Tangentially related: how does one get their foot in the door at a medical software company? Is there a big market for such professionals?

pram 8y ago
Learn MUMPS and move to Wisconsin.
- walshemj 8y ago
  Is MUMPS still a thing :-) maybe I ought to dust of my RT-11 / FORTRAN skills off
  - prepend 8y ago
    The biggest EMR company was Epic and uses Mumps [0]. However, they are losing share to Center and others [1], so this formula will only work for a little while longer. [0] https://news.ycombinator.com/item?id=13860937 https://news.ycombinator.com/item?id=13860937 [1] https://ehrintelligence.com/news/cerner-epic-mckesson-among-top-5-global-health-it-vendors https://ehrintelligence.com/news/cerner-epic-mckesson-among-...
    - sergers 8y ago
      Epic still dominates the EHR space. They have deployments slotted for 3 years. If you have ever worked with epic, they offer "managed" solutions, which you either do the epic designed way, or don't go epic... They have enough market share and business to dictate how things will work in a hospital env
tjr225 8y ago
I was recruited right out of college to work at Cerner. Its probably still fairly trivial to get a job there, but you have to live in Kansas City.
dbs 8y ago
You get at the door of companies in the healthcare space that need to get up to their game and tell them what you can do for them. That's what I did.

phren0logy 8y ago

Oh, I agree if it doesn't work it should be tossed aside. My concern is that given the potential usefulness demonstrated in so many other domains, I suspect that many of the tech failures in healthcare are due to the fact that it's too expensive to take the time to get it right.

mlthoughts2018 8y ago
I think this might also be supported by looking at some of the better funded / successfully exited health tech startups. I’m thinking of things like Flatiron Health, BenevolentAI, Quartet and Spreemo, and surely many more. At heart these companies seem to shy away from being “really about” medicine, and try to be a more watered down data company, looking at health records, social media, and other data to provide recommendation, service matching, comorbidity analysis, hospitalist tools, cost tools, assistance program allocation tools. I’ve been very interested in ML applications in health, but a lot of the businesses that seem capable of getting funding seem like they are super light / elementary on the statistical modeling side, and the value add is just a claim to modernize crappy claims data or unify a bunch of previously disparate health data sources. Maybe more advanced use of modeling will come later, though I am skeptical just given the way that once big enterprise customers dig hooks into essentially consulting services for health records, they won’t let go. Another thing that creeps me out is when you see e.g. an ex Palantir board member joining health record data companies. It’s not hard to understand why a big insurance company, or at worst even government agencies, want decision tools on top of huge stores of health record data. Do we really benefit from some super new / unproven startup building up that data set & tooling?

roymurdock 8y ago

Theranos is a counterexample. Probably would have been outed as fraudulent if they tried to enter a more safety-critical space (ventilators, pumps, injectors, etc.), but still a bad sign for medical diagnostic industry that it got so much funding and so far along, inking huge deals before collapsing. Perhaps some good will come if it makes others in the space more cautious in the short run Next industry to keep an eye on (many are already) is automotive. There are a lot of vendors trying to put a lot of software/connectivity into the next generation of cars. Lots of regulation but I will opt for a dumb car over new smart stuff any day. Same with a lot of different traditionally embedded systems that are now being connected to the internet

abakker 8y ago
Arguably, the big deals and optimism indicate that there is a lot of pent up demand for successful things, and people are willing to take some risk to get solutions. It’s not all doom and gloom, solutions just have to work.
- sk5t 8y ago
  Conversely, I read Theranos's progress as evidence of cynical bets that the government/military could likely be directed into pumping billions of USD into the company regardless of its merit.
  - abakker 8y ago
    You could certainly draw that conclusion from its original board. I did as well. Still, I think the point stands that people want healthcare to work, and will pay for it. It just so happens that it takes a lot of effort to improve the system since what we have is pretty good. (Not to say it doesn’t have problems)
nimish 8y ago
No, Theranos started to fail as soon as anyone with real medical expertise without a stake in the company started to probe it. Theranos was extremely successful at fleecing VCs and embarrassing political board members. It's a good thing that it got so far, because it proves just how bad VCs are at evaluating successful companies and how much of the funding game is who you know.

megaman22 8y ago

> The word from our engineers was that Watson was designed to sell commercials, not be a useful tool in a problem space This is the number one thing to remember about Watson. I've worked with teams within IBM that spent years and thousands upon thousands of man-hours trying to get it to be useful in a fairly well-defined, constrained use-case. Eventually they bailed and went to what was essentially a simple decision tree. It hits it out of the park on the sexy buzz-words, but I'm ready to curse the whole field for giving customers wildly inflated expectations.

Spooky23 8y ago
I worked on a project that did a little Watson work as well. Same result as you described. The shame is decision tree based expert systems can be pretty powerful, but the tools these guys were using were not that great.
- davak 8y ago
  I am a nerdy physician desperate to work with somebody on decision tree based expert systems. Are there a bunch of existing projects that I’m just missing?
  - gtmb 8y ago
    Well, I just started an open source project to gather the data for such systems. It is starting as a database/graph of symptoms and conditions. But the vision is bigger than that. https://github.com/gafmgafm/med https://github.com/gafmgafm/med There are similar products but they are not open source, such as: http://apimedic.com/ http://apimedic.com/ and http://www.diseasesdatabase.com/ http://www.diseasesdatabase.com/
    - mindcrime 8y ago
      I've been doing a lot of work lately with automated Abductive Inference, and medical diagnosis has been cited as one of the key uses of same. Perhaps there could be some opportunity to collaborate (everything I do is open source as well, btw). If you guys would like to talk, feel free to hit me up at prhodes@fogbeam.com

sergers 8y ago

I had posted this previously in a different thread, but it's all smoke and mirrors. I work in the diagnostic imaging space, we are working on integrating AI for assisted diagnosis or screening. It's a several-year road map for some base function. We had a top sales executive leave for IBM Watson Healthcare division about a year ago, was very proud to be part of what Watson was going to bring to healthcare He's been working back at my company, couldn't stand selling lies (and wasn't making much commission with canceled contracts). From a named company more impressed with what Google deepmind has done with NHS