7 ms·

I'm an MD, and I dabble in programming and AI/DL. While I don't really care much about Watson in particular, the pervasiveness of the "enter healthcare = death"

I'm an MD, and I dabble in programming and AI/DL. While I don't really care much about Watson in particular, the pervasiveness of the "enter healthcare = death" equation is worrisome to me. And it should worry you, too. There's a lot of potential for good applications of this type of tech in healthcare, but the barriers to actually getting it off the ground strangle it in the crib.

bitwize 8y ago
I think this has more to do with the fact that Watson sucks than anything else, although it may be a harbinger for other efforts to sprinkle magic machine-learning pixie dust on complex problems in the hope for a solution.
- phren0logy 8y ago
  In this case, you are probably right. But it seems to happen over and over again suggesting that the barriers to innovation are higher than they should be.
  - serioussecurity 8y ago
    Yeah, to be clear, Watson failing is good news. This is marketing people losing their minds because they didn't understand the products they were selling, and hospitals failing to do due diligence. I've talked to literally dozens of people / companies / academics running at problems in healthcare using deep learning and large data sets, and the recurrent theme is cowboys who pretty consistently fail to follow basic practices of experimental rigor. ML is frankly an area where more rigorous and clear regulation is desperately needed, because the potential to cut corners / cheat is so easy and the challenges of vetting these systems are specialized enough that most organizations won't successfully be able to catch fairly basic experimental mistakes.
ISL 8y ago
Many of those barriers have historical reasons for existing -- which do you think could be lowered without risking patient health (or yielding sufficient benefit to be worth some added risk)?
- phren0logy 8y ago
  Probably. The following applies to the US: It seems to me that some of the biggest barriers involve getting access to enough data to make a good training set. Some of this is related to HIPAA and related privacy laws, which are in place for a good reason but are still a major barrier. The other big factor is the fragmentation of the data across different, non-interoperable, non-standard formats from different vendors who have intentionally made interoperability difficult. That part bothers me much more. Many patients have data spread across a dozen or more paper charts, lab systems, EMRs, pharmacy databases, etc. As with most data science tasks, the "data munging" is the hardest part. I care about this mostly from the perspective of treating patients. Beyond training sets for AI, I need those records for the same reason: to make appropriate and informed treatment decisions. From the outside looking in, it's the kind of situation that begs companies to over-promise on results to get enough money to even give it a shot.
  - ISL 8y ago
    As a patient, I'd love to be able to archive/hold my own comprehensive copy of all of my health records.
  - xoail 8y ago
    I am working on a medical dictation app and the lack of any interoperability between 1000s of EHRs is mind boggling to me. There is absolutely no standards and like you mentioned its completely deliberate. Working through these platforms over the years has made me less and less optimistic about having a better health care system in the future.
    - sergers 8y ago
      Well FHIR is making its way, that looks positive, but no mainstream EHR has adopted it really. May I ask what your target users are? I sell alot of nuance PS360, and fluency
  - aaavl2821 8y ago
    Unfortunately, from my experience the biggest barriers to sharing data are very deeply embedded incentives. I was at a talk recently by an executive in charge of managing data for a massive health system. Think tens of millions of patients. He said they dont share data because if people learn their real quality metrics it would give payers, the public, anyone with an interest that isnt aligned w the hospital system ammunition against them. Have heard this sentiment echoed by many. Open data is their enemy because then patients could choose based on outcomes, not on market power or how nice the lobbies are The EMRs designed a closed garden into their systems DNA. If data is shared freely then it's easy to switch systems, and they will have to compete based on cost and quality and couldn't charge hundreds of millions for installations and maintenance contracts Call me cynical but this is all based on conversations with people well placed in the industry
Thriptic 8y ago
The two massive barriers to entry are regulatory uncertainty and sales difficulties Regulations: Up until recently, the regulatory landscape for medical software was extremely confusing. People were sitting on the sidelines waiting for FDA to figure out how to deal with machine learning, agile development, app stores, blah blah. Startups that wanted to jump in were staring at yearly 100K+ regulatory consultant bills. FDA is now actively de-regulating the bulk of medical software, and will create a regulatory kit which should better explain the regulations in an actionable way (instead of "create a design history file" without much explanation of wtf that means in practice). This should ease pains on that front Sales: This is where the big pain is. Getting paid for shit in healthcare is just too damn hard unless your product is on the hospital operations side of things. Getting an FDA approval (assuming it's required) DOES NOT mean one of the thousands of insurers will decide to reimburse you for your product; many require multiple clinical trials for reimbursement purposes; and many will decide to stop covering your product for stupid reasons (ie some editorial in the "No One Reads This Journal of Nonsense" said it was unclear the product was effective). Good luck getting patients to pay for anything, as they expect all products to be covered by insurance. Good luck convincing doctors they should use your product, as they are already overwhelmed with data entry / shitty UIs and don't want to perform any additional clicks or workflow changes.
- phren0logy 8y ago
  Glad to hear things are improving on the regulations front. Sales is going to be hard for a while, but I think that there are some markets where this is less of an issue, as it's more cash-based anyway. It makes me sad, though, that a lot of the population-based preventative stuff for which AI would be a great fit is unlikely to be a financially appealing investment.
- killjoywashere 8y ago
  Sales: skip it. Cardinal, Leica, Stryker, and others already have sales workforces. Cut your deal with one of them instead.
- criley2 8y ago
  The FDA is not the primary source of regulation in medical software, it's CMS. And while CMS regulations do not apply theoretically to private insurance and private practice, Medicare and Medicaid represent such a large segment of the healthcare economy that private insurers tend to use similar expectations as CMS regs. Also, software which cannot meet CMS reg and cannot be used for Medicare/Medicaid patients is a non-starter for the majority of the healthcare industry. The idea that our medical software space would be mostly deregulated is a rather horrifying thing in and of itself. I don't want my father being cared for using beta, buggy, software. I have high expectations for quality and risk management from healthcare software that I do not have for some agile beta video game or social network. I think your "bulk deregulation" comment is rather wrong. If you are coding in this space and are not INTIMATELY familiar with the concept of PHI and HIPAA, then you will be writing software which violates core regulations in this space. Your concerns on the sales side also do not mention "RISK" even once. You don't speak even a little bit of our healthcare space. Those doctors want risk management. What happens if/when the software screws up? Whose fault is it that the software is literally response for killing patients? You also don't mention what doctors care about: does it meet CMS regs? Does it meet state regs? Will I be able to submit my obscure XYZ state form? Will I be able to submit to BCBS, Aetna, Medicare? Will they accept? Etc etc. You will be writing custom software for customers all over, because the regulations that states and even cities like NYC might impose could be imposed on a single customer of yours only. I think people who want to disrupt healthcare forget that healthcare IT is literally life or death. Your bug isn't just an annoyance, it could be the thing that ends someones life. Regulations aren't just a barrier to agile development, they're also a life saving tool to ensure that risk management and data privacy are established in all products that get used on patients in our country.
criley2 8y ago
I work for a medical software company which evaluated Watson for use in our diagnostic product. It should be seen as a Good Thing that medical is a Serious Field where Non-Serious Ideas get run through the ringer and fail. And that's what happened with Watson. The word from our engineers was that Watson was designed to sell commercials, not be a useful tool in a problem space like differential diagnosis or drug reactions. I think this is good. Financial has the same reputation. You can't just use agile to bust out a minimum viable medical or financial product. You will be chewed up and spit out. And rightfully so. Because when it comes to our health, or our money and our investments, we don't have tolerance for failure. OK, my social network double posted my communication, annoying but it's just a communication. However "my financial tool just double posted a transaction", or "my prescriptions tool just doubled the dose of a medicine" -- these are not OK states at all. I wish Watson lived up to the hype, but sadly, our experience was that it does not. So it failing out of Healthcare isn't a surprise to us, nor is it a bad thing.
- yosyp 8y ago
  Tangentially related: how does one get their foot in the door at a medical software company? Is there a big market for such professionals?
  - pram 8y ago
    Learn MUMPS and move to Wisconsin.
    - walshemj 8y ago
      Is MUMPS still a thing :-) maybe I ought to dust of my RT-11 / FORTRAN skills off
      - prepend 8y ago
        The biggest EMR company was Epic and uses Mumps [0]. However, they are losing share to Center and others [1], so this formula will only work for a little while longer. [0] https://news.ycombinator.com/item?id=13860937 https://news.ycombinator.com/item?id=13860937 [1] https://ehrintelligence.com/news/cerner-epic-mckesson-among-top-5-global-health-it-vendors https://ehrintelligence.com/news/cerner-epic-mckesson-among-...

bitwize 8y ago

I think this has more to do with the fact that Watson sucks than anything else, although it may be a harbinger for other efforts to sprinkle magic machine-learning pixie dust on complex problems in the hope for a solution.

phren0logy 8y ago
In this case, you are probably right. But it seems to happen over and over again suggesting that the barriers to innovation are higher than they should be.
- serioussecurity 8y ago
  Yeah, to be clear, Watson failing is good news. This is marketing people losing their minds because they didn't understand the products they were selling, and hospitals failing to do due diligence. I've talked to literally dozens of people / companies / academics running at problems in healthcare using deep learning and large data sets, and the recurrent theme is cowboys who pretty consistently fail to follow basic practices of experimental rigor. ML is frankly an area where more rigorous and clear regulation is desperately needed, because the potential to cut corners / cheat is so easy and the challenges of vetting these systems are specialized enough that most organizations won't successfully be able to catch fairly basic experimental mistakes.

ISL 8y ago

Many of those barriers have historical reasons for existing -- which do you think could be lowered without risking patient health (or yielding sufficient benefit to be worth some added risk)?

phren0logy 8y ago
Probably. The following applies to the US: It seems to me that some of the biggest barriers involve getting access to enough data to make a good training set. Some of this is related to HIPAA and related privacy laws, which are in place for a good reason but are still a major barrier. The other big factor is the fragmentation of the data across different, non-interoperable, non-standard formats from different vendors who have intentionally made interoperability difficult. That part bothers me much more. Many patients have data spread across a dozen or more paper charts, lab systems, EMRs, pharmacy databases, etc. As with most data science tasks, the "data munging" is the hardest part. I care about this mostly from the perspective of treating patients. Beyond training sets for AI, I need those records for the same reason: to make appropriate and informed treatment decisions. From the outside looking in, it's the kind of situation that begs companies to over-promise on results to get enough money to even give it a shot.
- ISL 8y ago
  As a patient, I'd love to be able to archive/hold my own comprehensive copy of all of my health records.
- xoail 8y ago
  I am working on a medical dictation app and the lack of any interoperability between 1000s of EHRs is mind boggling to me. There is absolutely no standards and like you mentioned its completely deliberate. Working through these platforms over the years has made me less and less optimistic about having a better health care system in the future.
  - sergers 8y ago
    Well FHIR is making its way, that looks positive, but no mainstream EHR has adopted it really. May I ask what your target users are? I sell alot of nuance PS360, and fluency
- aaavl2821 8y ago
  Unfortunately, from my experience the biggest barriers to sharing data are very deeply embedded incentives. I was at a talk recently by an executive in charge of managing data for a massive health system. Think tens of millions of patients. He said they dont share data because if people learn their real quality metrics it would give payers, the public, anyone with an interest that isnt aligned w the hospital system ammunition against them. Have heard this sentiment echoed by many. Open data is their enemy because then patients could choose based on outcomes, not on market power or how nice the lobbies are The EMRs designed a closed garden into their systems DNA. If data is shared freely then it's easy to switch systems, and they will have to compete based on cost and quality and couldn't charge hundreds of millions for installations and maintenance contracts Call me cynical but this is all based on conversations with people well placed in the industry

Thriptic 8y ago

The two massive barriers to entry are regulatory uncertainty and sales difficulties Regulations: Up until recently, the regulatory landscape for medical software was extremely confusing. People were sitting on the sidelines waiting for FDA to figure out how to deal with machine learning, agile development, app stores, blah blah. Startups that wanted to jump in were staring at yearly 100K+ regulatory consultant bills. FDA is now actively de-regulating the bulk of medical software, and will create a regulatory kit which should better explain the regulations in an actionable way (instead of "create a design history file" without much explanation of wtf that means in practice). This should ease pains on that front Sales: This is where the big pain is. Getting paid for shit in healthcare is just too damn hard unless your product is on the hospital operations side of things. Getting an FDA approval (assuming it's required) DOES NOT mean one of the thousands of insurers will decide to reimburse you for your product; many require multiple clinical trials for reimbursement purposes; and many will decide to stop covering your product for stupid reasons (ie some editorial in the "No One Reads This Journal of Nonsense" said it was unclear the product was effective). Good luck getting patients to pay for anything, as they expect all products to be covered by insurance. Good luck convincing doctors they should use your product, as they are already overwhelmed with data entry / shitty UIs and don't want to perform any additional clicks or workflow changes.

phren0logy 8y ago
Glad to hear things are improving on the regulations front. Sales is going to be hard for a while, but I think that there are some markets where this is less of an issue, as it's more cash-based anyway. It makes me sad, though, that a lot of the population-based preventative stuff for which AI would be a great fit is unlikely to be a financially appealing investment.
killjoywashere 8y ago
Sales: skip it. Cardinal, Leica, Stryker, and others already have sales workforces. Cut your deal with one of them instead.
criley2 8y ago
The FDA is not the primary source of regulation in medical software, it's CMS. And while CMS regulations do not apply theoretically to private insurance and private practice, Medicare and Medicaid represent such a large segment of the healthcare economy that private insurers tend to use similar expectations as CMS regs. Also, software which cannot meet CMS reg and cannot be used for Medicare/Medicaid patients is a non-starter for the majority of the healthcare industry. The idea that our medical software space would be mostly deregulated is a rather horrifying thing in and of itself. I don't want my father being cared for using beta, buggy, software. I have high expectations for quality and risk management from healthcare software that I do not have for some agile beta video game or social network. I think your "bulk deregulation" comment is rather wrong. If you are coding in this space and are not INTIMATELY familiar with the concept of PHI and HIPAA, then you will be writing software which violates core regulations in this space. Your concerns on the sales side also do not mention "RISK" even once. You don't speak even a little bit of our healthcare space. Those doctors want risk management. What happens if/when the software screws up? Whose fault is it that the software is literally response for killing patients? You also don't mention what doctors care about: does it meet CMS regs? Does it meet state regs? Will I be able to submit my obscure XYZ state form? Will I be able to submit to BCBS, Aetna, Medicare? Will they accept? Etc etc. You will be writing custom software for customers all over, because the regulations that states and even cities like NYC might impose could be imposed on a single customer of yours only. I think people who want to disrupt healthcare forget that healthcare IT is literally life or death. Your bug isn't just an annoyance, it could be the thing that ends someones life. Regulations aren't just a barrier to agile development, they're also a life saving tool to ensure that risk management and data privacy are established in all products that get used on patients in our country.

criley2 8y ago

I work for a medical software company which evaluated Watson for use in our diagnostic product. It should be seen as a Good Thing that medical is a Serious Field where Non-Serious Ideas get run through the ringer and fail. And that's what happened with Watson. The word from our engineers was that Watson was designed to sell commercials, not be a useful tool in a problem space like differential diagnosis or drug reactions. I think this is good. Financial has the same reputation. You can't just use agile to bust out a minimum viable medical or financial product. You will be chewed up and spit out. And rightfully so. Because when it comes to our health, or our money and our investments, we don't have tolerance for failure. OK, my social network double posted my communication, annoying but it's just a communication. However "my financial tool just double posted a transaction", or "my prescriptions tool just doubled the dose of a medicine" -- these are not OK states at all. I wish Watson lived up to the hype, but sadly, our experience was that it does not. So it failing out of Healthcare isn't a surprise to us, nor is it a bad thing.

yosyp 8y ago
Tangentially related: how does one get their foot in the door at a medical software company? Is there a big market for such professionals?
- pram 8y ago
  Learn MUMPS and move to Wisconsin.
  - walshemj 8y ago
    Is MUMPS still a thing :-) maybe I ought to dust of my RT-11 / FORTRAN skills off
    - prepend 8y ago
      The biggest EMR company was Epic and uses Mumps [0]. However, they are losing share to Center and others [1], so this formula will only work for a little while longer. [0] https://news.ycombinator.com/item?id=13860937 https://news.ycombinator.com/item?id=13860937 [1] https://ehrintelligence.com/news/cerner-epic-mckesson-among-top-5-global-health-it-vendors https://ehrintelligence.com/news/cerner-epic-mckesson-among-...