7 ms·

Having worked with data from EMR systems and having worked at a large EMR software development shop myself, and now using deep learning at work quite a bit for

Having worked with data from EMR systems and having worked at a large EMR software development shop myself, and now using deep learning at work quite a bit for the past few years, I'm inclined to agree. This title is somewhat click bait though, because the fault is really with EMR systems and (esp) the American Healthcare system, not deep learning. The entire system is designed around billing and decisions are made my hospital and insurance executives that are generally not technical. There is no incentive to clean up the system or work on a well structured open protocol for interop the same way there is in say banking. Plus, the author gives some good examples like pulse ox%, doctors and nurses are not at all concerned with or trained to record data in a way that makes sense to use programmatically. They're typically thinking only as if they're recording it for another human to read. Deep learning could probably be quite useful in the medical field, but we won't know until someone comes along and disrupts the system top to bottom similar to how Tesla has done with not only manufacturing, but the sales process and shirking the dealership model. This would probably look something like Forward[1], but with a crazy amount of funding, so that insurance companies and billing codes could be ignored entirely. [1]: https://goforward.com/p/home https://goforward.com/p/home

ulkesh 5y ago
Yes, exactly. Hospital data is atrocious, on every single level. Duplicate data are everywhere across multiple systems. Hospitals move excruciatingly slow to do anything technical. And, very few people seem to ever have any real understanding of what is going on with their systems, they tend to only know the user interface and have to rely on their vendor support (Epic, Cerner, etc) for anything beyond that. I work for a company that I was hoping would be such a disruption point for hospitals (at least in some small way), but instead they decided that it's just too difficult to get hospitals to do much of anything, so we effectively knuckle under -- creating numerous integration points, making more and more copies of data. The only way this will change is if a large enough player creates direct competition in the EHR/EMR business, with these kinds of data-oriented models in mind, all the while creating a system that, top-to-bottom, is better for both the user, the administrator, and the technical staff. Current players in this space have very little incentive to make their products better. And it reminds me of a quote from Tron Legacy: "Given the prices we charge to students and schools, what sort of improvements have been made in Flynn... I mean, um, ENCOM OS-12?"..."This year we put a "12" on the box."
- divbzero 5y ago
  Part of the challenge might be the limited number of individuals with expertise in both medicine and technology. Each of those fields are quite specialized and require time to build an understanding of the subject matter.
- nradov 5y ago
  It would take billions of dollars and many years to build a new general purpose EHR from scratch. This isn't an industry where a start-up can launch an MVP and get sales to early adopters looking for a competitive advantage. Every major provider organization already has an EHR, and usually an ONC certified one. A new offering has to be at least as good as existing products in every way in order to get any sales. It has to be a 100% solution out of the gate. Outside of general purpose comprehensive EHRs there is still opportunity for new niche market offerings targeting limited medical specialties or types of facilities. For example if you wanted to target, let's say, just dialysis centers or just psychiatrists then the hurdle would be much lower.
  - vanviegen 5y ago
    A bit of inspired legislation could solve all of these problems.
    - lowbloodsugar 5y ago
      Sure. That's where the first few billions of dollars and several years are going to have to go.
      - voakbasda 5y ago
        And would you trust anyone willing and able to go through that process? The lobbying and political game is filled with sociopaths that would sell out the public for a hot lunch.
    - yold__ 5y ago
      Not sure if you are being sarcastic, but that's precisely what the ACA did. It created giant incentives for hospitals to modernize their medical record systems, which many did.
modeless 5y ago
> Plus, the author gives some good examples like pulse ox%, doctors and nurses are not at all concerned with or trained to record data in a way that makes sense to use programmatically. They're typically thinking only as if they're recording it for another human to read. The unstructured doctor's notes may be where the best signal lies. Thet should be mostly uncorrupted by the billing/process related trash. Deep learning does not need data to be formatted to be read "programmatically", in fact it shines most on data that isn't.
- yarky 5y ago
  > The unstructured doctor's notes may be where the best signal lies. Or where the raw data is corrupted ...
  - withinboredom 5y ago
    Doctors notes: removed 2 polyps Billing: unremarkable endoscopy Which is true? (Based on a true story)
    - pc86 5y ago
      They're both true of course. Polyps get removed all the time, it is unremarkable in a medical sense, and probably not enough to even warrant its own billing code, otherwise it certainly would have been listed.
      - withinboredom 5y ago
        Ah yeah, that’s good point. In either case, the true story ended with no polyps and it was a transcription error.
      - danachow 5y ago
        No this is not true at all. Procedural documentation in the US is exquisitely detailed both for medical-legal reasons and billing. Any polyp in the colon will be grossly described, it’s approximate position and if any intervention or send out testing is being performed. And as an internist - no colon polyp is medically unremarkable.
nradov 5y ago
Implementation guides built on HL7 FHIR are well structured open protocols for interoperability. Several of the largest EHR vendors actively participate in defining those standards, and have built them into products. Of course that doesn't solve the garbage in / garbage out problem. If users configure the software incorrectly or don't enter the right data then then you don't have much to work with. In some cases NLP technology can be used to convert unstructured chart notes into coded clinical concepts. That can work well enough for research and analytics, but isn't accurate enough to use for direct patient care without human review.
- shakna 5y ago
  HL7. Well-structured. Pick one. The HL7 spec might be a standard, with a specification, but it is not a thing for well-structured data. For example, FHIR allows you to use a standardised shorthand, with a formal grammar. [0] That grammar is well defined... It also happens not be a regular grammar, and it is entirely possible to construct something that will be both valid and undecidable, ala Perl. Infinite parsing and expansion on finite data. [0] https://hl7.org/fhir/uv/shorthand/STU1/reference.html#appendix-formal-grammar https://hl7.org/fhir/uv/shorthand/STU1/reference.html#append...
  - specialist 5y ago
    Yup. After a few years of trying to follow the rules, I just decided to use data scrapping techniques.
  - paulmd 5y ago
    Yes, think of HL7 like XML or JSON. It's a grammar/syntax, it does provide some domain-specific conceptual mapping but you can code the same thing lots of different ways. Emitting a well-formed HL7 message doesn't mean that anybody can actually ingest them. There's also a fair amount of instance-specific coding and going to/from that is always a pain.
    - nradov 5y ago
      I wouldn't think of HL7 like XML or JSON, it's higher level than that. HL7 is a standards development organization which publishes multiple separate major standards. Those standards define data models which can be encoded in various different wire formats. V2 Messaging can be encoded as ER7 or (rarely) XML. CDA can only be encoded as XML. FHIR can be encoded as JSON or XML. If you want recipients to be able to ingest your messages or documents then it's crucial for the sender and recipient to mutually agree to conform to a specific versioned Implementation Guide. If both parties actually follow the IG then it eliminates the need for instance specific coding, but of course some organizations still have quality problems.
deleted 5y ago
[deleted]
midjji 5y ago
Having worked with otherwise decent Swedish public healthcare systems, I can't tell you if its as bad, just that it is absolutely awful. I would never think to try and use anything they provided as basis for a dataset.
BeFlatXIII 5y ago
> doctors and nurses are not at all concerned with or trained to record data in a way that makes sense to use programmatically. They're typically thinking only as if they're recording it for another human to read. The flip side of this is that data entered for programmatic reading often isn't very useful for future humans to review. Automatically-generated doctor's notes obscure information in a way that "HEART ATTACK" circled in red ink on a paper chart does not.
divbzero 5y ago
Wouldn’t heath systems like Kaiser Permanente that are fully integrated (both insurer and provider) have the right incentives to disrupt their own processes in favor of something better?
- nradov 5y ago
  What sort of incentive were you thinking of? KP is one of the largest Epic customers. They don't have claims as such, but still have similar internal processes to prevent waste and determine how much to charge their customers.
  - dragonwriter 5y ago
    > KP is one of the largest Epic customers. They don't have claims as such Yes, they do (whether you are talking about KP-as-provider-network submitting claims to non-KP insurers, or KP-as-insurer adjudicating claims from non-KP providers.)
    - nradov 5y ago
      I was addressing the comment from @divbzero about integrated systems. Out of network claims are a separate issue.
  - divbzero 5y ago
    I was thinking about incentives around cost of care. KP could switch from Epic to an alternative if it enables deep learning that reduces costs in the long run. Reducing costs is aligned with KP’s incentives because they are not only the provider but also the payer for medical care. This is different from most medical providers which OP suggests have an incentive to be “cost maximizers” to increase reimbursements from payers.
not2b 5y ago
"This title is somewhat click bait though, because the fault is really with EMR systems". The title is entirely accurate: it specifies "on electronic medical records" and effectively demonstrates the thesis in the title. And yes, you're right that the problem is in the data. No matter, it's still true. Everything with an even mildly provocative title is accused of being "clickbait" on HN. I wish people would reserve that criticism for actively deceptive titles, or titles that promise something that is not delivered.
pkaye 5y ago
So does that mean it might be of use to non American healthcare systems which have less to deal with billing?
toss1 5y ago
>>This title is somewhat click bait though, because the fault is really with EMR systems and (esp) the American Healthcare system, not deep learning. Right Data-mush is the cause That's my official term for it. Data that looks fantastic in format, but just the slightest peek under the covers reveals that every data-entry person is either freelancing and entering whatever is easiest/sorta-makes-sense for them, or under pressure to skew things, or the definitions of each code (and the cases where it does/doesn't apply) are ill-defined, or etc. etc. etc. and on top of this, we have the medical system insurer-provider adversarial relationship covered so well in the article. The result is a toxic brew of definition drift, unintentional errors, and intentional errors. It is not just the fringes and sub-one-percent of the edge cases, it is rotten to the core. Basically the entire data set is a complete lie. You may have the most perfect AI, but it literally has no chance against that toxic data swamp. Once again, the only winning move is to not play.
deleted 5y ago
[deleted]

ulkesh 5y ago

Yes, exactly. Hospital data is atrocious, on every single level. Duplicate data are everywhere across multiple systems. Hospitals move excruciatingly slow to do anything technical. And, very few people seem to ever have any real understanding of what is going on with their systems, they tend to only know the user interface and have to rely on their vendor support (Epic, Cerner, etc) for anything beyond that. I work for a company that I was hoping would be such a disruption point for hospitals (at least in some small way), but instead they decided that it's just too difficult to get hospitals to do much of anything, so we effectively knuckle under -- creating numerous integration points, making more and more copies of data. The only way this will change is if a large enough player creates direct competition in the EHR/EMR business, with these kinds of data-oriented models in mind, all the while creating a system that, top-to-bottom, is better for both the user, the administrator, and the technical staff. Current players in this space have very little incentive to make their products better. And it reminds me of a quote from Tron Legacy: "Given the prices we charge to students and schools, what sort of improvements have been made in Flynn... I mean, um, ENCOM OS-12?"..."This year we put a "12" on the box."

divbzero 5y ago
Part of the challenge might be the limited number of individuals with expertise in both medicine and technology. Each of those fields are quite specialized and require time to build an understanding of the subject matter.
nradov 5y ago
It would take billions of dollars and many years to build a new general purpose EHR from scratch. This isn't an industry where a start-up can launch an MVP and get sales to early adopters looking for a competitive advantage. Every major provider organization already has an EHR, and usually an ONC certified one. A new offering has to be at least as good as existing products in every way in order to get any sales. It has to be a 100% solution out of the gate. Outside of general purpose comprehensive EHRs there is still opportunity for new niche market offerings targeting limited medical specialties or types of facilities. For example if you wanted to target, let's say, just dialysis centers or just psychiatrists then the hurdle would be much lower.
- vanviegen 5y ago
  A bit of inspired legislation could solve all of these problems.
  - lowbloodsugar 5y ago
    Sure. That's where the first few billions of dollars and several years are going to have to go.
    - voakbasda 5y ago
      And would you trust anyone willing and able to go through that process? The lobbying and political game is filled with sociopaths that would sell out the public for a hot lunch.
  - yold__ 5y ago
    Not sure if you are being sarcastic, but that's precisely what the ACA did. It created giant incentives for hospitals to modernize their medical record systems, which many did.

modeless 5y ago

> Plus, the author gives some good examples like pulse ox%, doctors and nurses are not at all concerned with or trained to record data in a way that makes sense to use programmatically. They're typically thinking only as if they're recording it for another human to read. The unstructured doctor's notes may be where the best signal lies. Thet should be mostly uncorrupted by the billing/process related trash. Deep learning does not need data to be formatted to be read "programmatically", in fact it shines most on data that isn't.

yarky 5y ago
> The unstructured doctor's notes may be where the best signal lies. Or where the raw data is corrupted ...
- withinboredom 5y ago
  Doctors notes: removed 2 polyps Billing: unremarkable endoscopy Which is true? (Based on a true story)
  - pc86 5y ago
    They're both true of course. Polyps get removed all the time, it is unremarkable in a medical sense, and probably not enough to even warrant its own billing code, otherwise it certainly would have been listed.
    - withinboredom 5y ago
      Ah yeah, that’s good point. In either case, the true story ended with no polyps and it was a transcription error.
    - danachow 5y ago
      No this is not true at all. Procedural documentation in the US is exquisitely detailed both for medical-legal reasons and billing. Any polyp in the colon will be grossly described, it’s approximate position and if any intervention or send out testing is being performed. And as an internist - no colon polyp is medically unremarkable.

nradov 5y ago

Implementation guides built on HL7 FHIR are well structured open protocols for interoperability. Several of the largest EHR vendors actively participate in defining those standards, and have built them into products. Of course that doesn't solve the garbage in / garbage out problem. If users configure the software incorrectly or don't enter the right data then then you don't have much to work with. In some cases NLP technology can be used to convert unstructured chart notes into coded clinical concepts. That can work well enough for research and analytics, but isn't accurate enough to use for direct patient care without human review.

shakna 5y ago
HL7. Well-structured. Pick one. The HL7 spec might be a standard, with a specification, but it is not a thing for well-structured data. For example, FHIR allows you to use a standardised shorthand, with a formal grammar. [0] That grammar is well defined... It also happens not be a regular grammar, and it is entirely possible to construct something that will be both valid and undecidable, ala Perl. Infinite parsing and expansion on finite data. [0] https://hl7.org/fhir/uv/shorthand/STU1/reference.html#appendix-formal-grammar https://hl7.org/fhir/uv/shorthand/STU1/reference.html#append...
- specialist 5y ago
  Yup. After a few years of trying to follow the rules, I just decided to use data scrapping techniques.
- paulmd 5y ago
  Yes, think of HL7 like XML or JSON. It's a grammar/syntax, it does provide some domain-specific conceptual mapping but you can code the same thing lots of different ways. Emitting a well-formed HL7 message doesn't mean that anybody can actually ingest them. There's also a fair amount of instance-specific coding and going to/from that is always a pain.
  - nradov 5y ago
    I wouldn't think of HL7 like XML or JSON, it's higher level than that. HL7 is a standards development organization which publishes multiple separate major standards. Those standards define data models which can be encoded in various different wire formats. V2 Messaging can be encoded as ER7 or (rarely) XML. CDA can only be encoded as XML. FHIR can be encoded as JSON or XML. If you want recipients to be able to ingest your messages or documents then it's crucial for the sender and recipient to mutually agree to conform to a specific versioned Implementation Guide. If both parties actually follow the IG then it eliminates the need for instance specific coding, but of course some organizations still have quality problems.

deleted 5y ago

[deleted]

midjji 5y ago

Having worked with otherwise decent Swedish public healthcare systems, I can't tell you if its as bad, just that it is absolutely awful. I would never think to try and use anything they provided as basis for a dataset.

BeFlatXIII 5y ago

> doctors and nurses are not at all concerned with or trained to record data in a way that makes sense to use programmatically. They're typically thinking only as if they're recording it for another human to read. The flip side of this is that data entered for programmatic reading often isn't very useful for future humans to review. Automatically-generated doctor's notes obscure information in a way that "HEART ATTACK" circled in red ink on a paper chart does not.

divbzero 5y ago

Wouldn’t heath systems like Kaiser Permanente that are fully integrated (both insurer and provider) have the right incentives to disrupt their own processes in favor of something better?

nradov 5y ago
What sort of incentive were you thinking of? KP is one of the largest Epic customers. They don't have claims as such, but still have similar internal processes to prevent waste and determine how much to charge their customers.
- dragonwriter 5y ago
  > KP is one of the largest Epic customers. They don't have claims as such Yes, they do (whether you are talking about KP-as-provider-network submitting claims to non-KP insurers, or KP-as-insurer adjudicating claims from non-KP providers.)
  - nradov 5y ago
    I was addressing the comment from @divbzero about integrated systems. Out of network claims are a separate issue.
- divbzero 5y ago
  I was thinking about incentives around cost of care. KP could switch from Epic to an alternative if it enables deep learning that reduces costs in the long run. Reducing costs is aligned with KP’s incentives because they are not only the provider but also the payer for medical care. This is different from most medical providers which OP suggests have an incentive to be “cost maximizers” to increase reimbursements from payers.

not2b 5y ago

"This title is somewhat click bait though, because the fault is really with EMR systems". The title is entirely accurate: it specifies "on electronic medical records" and effectively demonstrates the thesis in the title. And yes, you're right that the problem is in the data. No matter, it's still true. Everything with an even mildly provocative title is accused of being "clickbait" on HN. I wish people would reserve that criticism for actively deceptive titles, or titles that promise something that is not delivered.

pkaye 5y ago

So does that mean it might be of use to non American healthcare systems which have less to deal with billing?

toss1 5y ago

>>This title is somewhat click bait though, because the fault is really with EMR systems and (esp) the American Healthcare system, not deep learning. Right Data-mush is the cause That's my official term for it. Data that looks fantastic in format, but just the slightest peek under the covers reveals that every data-entry person is either freelancing and entering whatever is easiest/sorta-makes-sense for them, or under pressure to skew things, or the definitions of each code (and the cases where it does/doesn't apply) are ill-defined, or etc. etc. etc. and on top of this, we have the medical system insurer-provider adversarial relationship covered so well in the article. The result is a toxic brew of definition drift, unintentional errors, and intentional errors. It is not just the fringes and sub-one-percent of the edge cases, it is rotten to the core. Basically the entire data set is a complete lie. You may have the most perfect AI, but it literally has no chance against that toxic data swamp. Once again, the only winning move is to not play.

deleted 5y ago

[deleted]