6 ms·

How a tiny hospital used AI to lower costs and improve patient outcomes

109 points by jtsymonds 8y ago

trhway 8y ago
Reminded that episode from Greys Anatomy where they, using similar approach [manually, not AI though], discovered that pulling kidney surgery drainage couple days earlier (on the 3rd day instead of 5th) minimizes complications like fistula. The AI produced recommendations in the article doesn't seem to have any rationale other than statistical correlation ("topological data analysis"). Patients who had that care path did better - i hope they controlled for the possibility that those patients may have been the "better" ones to start with. >“After looking at the data, a cardiologist on the Pit Crew said, ‘Oh my, the Goldilocks group of doctors did less than I would have done but achieved as good or better outcomes. Therefore, when we add anything outside the CarePath, except regulatory requirements, we are adding cost without any benefit to the patient.’ For him, that was a ‘light bulb’ moment.” I hope there was another cardiologist near by when this one was having his light bulb moment. Overall i suppose it is a nice (by the metrics of healthcare IT ) sales push piece. I'm kind of wondering why such basic analytical practice hasn't been a routine for decades already ( may be because anybody able to do basic statistics would make much better money on wall street than being a lowly number cruncher at a dusty office in the hospital basement ... right to the moment of course when the thing gets renamed to AI and starts to be shiny and glorious and the hospitals start to pay good money for it). >Flagler expects to save $1,356.35 per pneumonia patient better than if i switch to Geyco.
- merpnderp 8y ago
  Unless the people were dying, the proof is in the vastly reduced readmittance rate.
tomohawk 8y ago
> "expects to save" Shortened stays, less expense, and fewer tests sound nice, but and independent verification after some period of time seems imperative. It also makes me wonder about some of the outlier patients. Those tests may be very beneficial for them. However, I'm sure doctors may feel pressure to not order them due to the cost savings.
- merpnderp 8y ago
  Likely hard to fake the vastly reduced number of readmittances.
  - riahi 8y ago
    As mentioned elsewhere in the comments, small/tiny hospitals rarely keep anything complicated and ship everything vaguely complex to tertiary/quaternary center.
maxxxxx 8y ago
I am glad that they are lowering costs but is this really about AI? Usually someone can sit down and quickly find ways to reduce costs but the problem is usually the power to actually implement them. It's like dealing with consultants. The company's employees already know what's wrong but don't have the power to change. Then consultants come in with CEO backing and suddenly change gets made. Consultants (or AI) get the credit but it's really about being to implement.
- merpnderp 8y ago
  The article cites a specific example of how a model spat out a positive relationship between pneumonia with existing COPD and the speed of starting nebulizer treatments, which they stated was non-obvious to staff.
- blang 8y ago
  only slightly related: https://www.cartalk.com/content/cowboy-and-consultant https://www.cartalk.com/content/cowboy-and-consultant
- boxspam 8y ago
  This is more of a puffy PR piece for the IT supplier. The methods they use are very advanced mathematics / graph theory. If we take the modern usage of the word "AI", then this qualifies. These methods are beyond what anyone can do in Excel. Usually data is so big and complex that a manual analysis takes months. It does not scale to sit down and find ways to reduce costs. Using unsupervised pattern detection does scale. About the consultants vs. engineers and the power to actually implement systems: Many IT systems that call themselves "AI", are not. They require tons of labeled data, a human making assumptions, lack justification and transparency, are not embedded properly in the business (lack UI, documentation, buy-in, data processing), and don't continuously learn. You need an entire ecosystem of processes and software tools to have the power to change (identifying problems is easy, of course engineers already know what's wrong with their daily work, if they could implement a solution, then a proper engineer would, but they really need a consultant and CEO backing to actually get something actionable).
joshgel 8y ago
As a physician who does lots of bioinformatics, this seems like a very optimistic article which is likely just overfitting the existing data. Patients don't come to the hospital saying "I have pneumonia". The come for a cough or a fever and it takes these tests to figure out the diagnosis. A small hospital like this probably doesn't have the data to really build a robust model that will help all its patients. Even major academic centers have relatively limited data for the number of permutations of health possibilities. Sure they have lots of pneumonia data, but how many cases of pneumonia do they have in cystic fibrosis patients who also have COPD? I certainly think AI can help cut costs, reduce over-testing and reduce length of stay, but decreasing length of stay by 2 days would be a shocking advance. (The average simple pneumonia LOS of stay for a medicare patient without co-morbidities* was 3.3 days and with major co-morbidities* was 5.7 days in 2017). [0] *as defined by medicare. [0] https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/FY2017-IPPS-Final-Rule-Home-Page-Items/FY2017-IPPS-Final-Rule-Tables.html https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Paymen...
- boxspam 8y ago
  > is likely just overfitting the existing data. I would be more careful with your assessment. Or you are overfitting to your limited 5-minute-understanding of what this does. Telling mathematicians and statisticians they have overfitted is akin to a grave professional insult (you are basically accusing them of foolish behavior at best, fraud/unethical behavior at worst. As a bioinformatician, you may be interested in these articles, which go a little deeper on the technology: https://www.nature.com/articles/srep01236 https://www.nature.com/articles/srep01236 https://www.sciencedirect.com/science/article/pii/S2405471216301831 https://www.sciencedirect.com/science/article/pii/S240547121... http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0126383 http://journals.plos.org/plosone/article?id=10.1371/journal.... > probably doesn't have the data to really build a robust model It is both possible to build robust models on small datasets (and that data is shared, or more abundant than you expect). > Even major academic centers have relatively limited data for the number of permutations of health possibilities. This is one of the problems they (the IT supplier) solved. They are able to build regulatory-proof models from 100.000's of permutations. Another, more recent, advance is in counterfactual analysis (what would have happened if this pneumonia patient also had COPD?).
bayesian_horse 8y ago
Sounds like someone is applying statistics with lower statistical thresholds and calling the method "AI" instead.
- larrydag 8y ago
  That's what I thought too. It sounds like basic data analysis. Nothing wrong with that because it brought good results. Why tag the project with AI?
  - reaperducer 8y ago
    Funding.
- totoglazer 8y ago
  I say neigh to their posterior confidence interval.
- mr_toad 8y ago
  Clustering & unsupervised machine learning are still considered to be branches of AI. Not everything is neural nets and DL.
  - bayesian_horse 8y ago
    Yes, of course, but this article seems to be neither unsupervised nor supervised learning, just some data analysis and statistics. It's not that it is completely different from AI, but you don't need the term "AI" to describe it, except if you want to hide the methodological weaknesses. They say they haven't really studied if the recommendations work in practice, it's just the prediction of a statistical model.
Justin_K 8y ago
HIT news is paid articles by the vendors. It's an industry rag and all their stories read the same, with the vendor name getting dropped about paragraph 3.
ballenf 8y ago
> It expects to save $1,356.35 per pneumonia patient in direct variable costs (35 percent savings) versus the status quo, while reducing length of stay by two days. The new sepsis pathway has also been deployed. So the headline was BS. This is the projected savings estimate used by the vendor, who totally coincidentally got a bump in their contract from 4 case studies a year to 12 on the basis of this totally objective guess.
SQL2219 8y ago
I don't understand how their readmission rates are so low at 2.9%, I was just checking some stats on that at the link below, typical readmission rates are 15-20%: https://data.medicare.gov/Hospital-Compare/Hospital-Readmission-Rates/92ps-fthr/data https://data.medicare.gov/Hospital-Compare/Hospital-Readmiss...
- Fomite 8y ago
  "Tiny hospitals" have low readmission rates, because the complicated patients with lots of readmissions get transferred to tertiary care centers. This also came up in things like Consumer Reports and their analysis of hospital acquired infection data. Small community hospitals with uncomplicated patients end up looking really good.
wolfi1 8y ago
when such terms as "unsupervised learning" are coined I doubt that there was something achieved, common sense wouldn't have achieved, too. As an AI-expert once said to me: unsupervised learning does not work

109 points by jtsymonds 8y ago

Reminded that episode from Greys Anatomy where they, using similar approach [manually, not AI though], discovered that pulling kidney surgery drainage couple days earlier (on the 3rd day instead of 5th) minimizes complications like fistula. The AI produced recommendations in the article doesn't seem to have any rationale other than statistical correlation ("topological data analysis"). Patients who had that care path did better - i hope they controlled for the possibility that those patients may have been the "better" ones to start with. >“After looking at the data, a cardiologist on the Pit Crew said, ‘Oh my, the Goldilocks group of doctors did less than I would have done but achieved as good or better outcomes. Therefore, when we add anything outside the CarePath, except regulatory requirements, we are adding cost without any benefit to the patient.’ For him, that was a ‘light bulb’ moment.” I hope there was another cardiologist near by when this one was having his light bulb moment. Overall i suppose it is a nice (by the metrics of healthcare IT ) sales push piece. I'm kind of wondering why such basic analytical practice hasn't been a routine for decades already ( may be because anybody able to do basic statistics would make much better money on wall street than being a lowly number cruncher at a dusty office in the hospital basement ... right to the moment of course when the thing gets renamed to AI and starts to be shiny and glorious and the hospitals start to pay good money for it). >Flagler expects to save $1,356.35 per pneumonia patient better than if i switch to Geyco.

merpnderp 8y ago
Unless the people were dying, the proof is in the vastly reduced readmittance rate.

tomohawk 8y ago

> "expects to save" Shortened stays, less expense, and fewer tests sound nice, but and independent verification after some period of time seems imperative. It also makes me wonder about some of the outlier patients. Those tests may be very beneficial for them. However, I'm sure doctors may feel pressure to not order them due to the cost savings.

merpnderp 8y ago
Likely hard to fake the vastly reduced number of readmittances.
- riahi 8y ago
  As mentioned elsewhere in the comments, small/tiny hospitals rarely keep anything complicated and ship everything vaguely complex to tertiary/quaternary center.

maxxxxx 8y ago

I am glad that they are lowering costs but is this really about AI? Usually someone can sit down and quickly find ways to reduce costs but the problem is usually the power to actually implement them. It's like dealing with consultants. The company's employees already know what's wrong but don't have the power to change. Then consultants come in with CEO backing and suddenly change gets made. Consultants (or AI) get the credit but it's really about being to implement.

merpnderp 8y ago
The article cites a specific example of how a model spat out a positive relationship between pneumonia with existing COPD and the speed of starting nebulizer treatments, which they stated was non-obvious to staff.
blang 8y ago
only slightly related: https://www.cartalk.com/content/cowboy-and-consultant https://www.cartalk.com/content/cowboy-and-consultant
boxspam 8y ago
This is more of a puffy PR piece for the IT supplier. The methods they use are very advanced mathematics / graph theory. If we take the modern usage of the word "AI", then this qualifies. These methods are beyond what anyone can do in Excel. Usually data is so big and complex that a manual analysis takes months. It does not scale to sit down and find ways to reduce costs. Using unsupervised pattern detection does scale. About the consultants vs. engineers and the power to actually implement systems: Many IT systems that call themselves "AI", are not. They require tons of labeled data, a human making assumptions, lack justification and transparency, are not embedded properly in the business (lack UI, documentation, buy-in, data processing), and don't continuously learn. You need an entire ecosystem of processes and software tools to have the power to change (identifying problems is easy, of course engineers already know what's wrong with their daily work, if they could implement a solution, then a proper engineer would, but they really need a consultant and CEO backing to actually get something actionable).

joshgel 8y ago

As a physician who does lots of bioinformatics, this seems like a very optimistic article which is likely just overfitting the existing data. Patients don't come to the hospital saying "I have pneumonia". The come for a cough or a fever and it takes these tests to figure out the diagnosis. A small hospital like this probably doesn't have the data to really build a robust model that will help all its patients. Even major academic centers have relatively limited data for the number of permutations of health possibilities. Sure they have lots of pneumonia data, but how many cases of pneumonia do they have in cystic fibrosis patients who also have COPD? I certainly think AI can help cut costs, reduce over-testing and reduce length of stay, but decreasing length of stay by 2 days would be a shocking advance. (The average simple pneumonia LOS of stay for a medicare patient without co-morbidities* was 3.3 days and with major co-morbidities* was 5.7 days in 2017). [0] *as defined by medicare. [0] https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/FY2017-IPPS-Final-Rule-Home-Page-Items/FY2017-IPPS-Final-Rule-Tables.html https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Paymen...

boxspam 8y ago
> is likely just overfitting the existing data. I would be more careful with your assessment. Or you are overfitting to your limited 5-minute-understanding of what this does. Telling mathematicians and statisticians they have overfitted is akin to a grave professional insult (you are basically accusing them of foolish behavior at best, fraud/unethical behavior at worst. As a bioinformatician, you may be interested in these articles, which go a little deeper on the technology: https://www.nature.com/articles/srep01236 https://www.nature.com/articles/srep01236 https://www.sciencedirect.com/science/article/pii/S2405471216301831 https://www.sciencedirect.com/science/article/pii/S240547121... http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0126383 http://journals.plos.org/plosone/article?id=10.1371/journal.... > probably doesn't have the data to really build a robust model It is both possible to build robust models on small datasets (and that data is shared, or more abundant than you expect). > Even major academic centers have relatively limited data for the number of permutations of health possibilities. This is one of the problems they (the IT supplier) solved. They are able to build regulatory-proof models from 100.000's of permutations. Another, more recent, advance is in counterfactual analysis (what would have happened if this pneumonia patient also had COPD?).

bayesian_horse 8y ago

Sounds like someone is applying statistics with lower statistical thresholds and calling the method "AI" instead.

larrydag 8y ago
That's what I thought too. It sounds like basic data analysis. Nothing wrong with that because it brought good results. Why tag the project with AI?
- reaperducer 8y ago
  Funding.
totoglazer 8y ago
I say neigh to their posterior confidence interval.
mr_toad 8y ago
Clustering & unsupervised machine learning are still considered to be branches of AI. Not everything is neural nets and DL.
- bayesian_horse 8y ago
  Yes, of course, but this article seems to be neither unsupervised nor supervised learning, just some data analysis and statistics. It's not that it is completely different from AI, but you don't need the term "AI" to describe it, except if you want to hide the methodological weaknesses. They say they haven't really studied if the recommendations work in practice, it's just the prediction of a statistical model.

Justin_K 8y ago

HIT news is paid articles by the vendors. It's an industry rag and all their stories read the same, with the vendor name getting dropped about paragraph 3.

ballenf 8y ago

> It expects to save $1,356.35 per pneumonia patient in direct variable costs (35 percent savings) versus the status quo, while reducing length of stay by two days. The new sepsis pathway has also been deployed. So the headline was BS. This is the projected savings estimate used by the vendor, who totally coincidentally got a bump in their contract from 4 case studies a year to 12 on the basis of this totally objective guess.

SQL2219 8y ago

I don't understand how their readmission rates are so low at 2.9%, I was just checking some stats on that at the link below, typical readmission rates are 15-20%: https://data.medicare.gov/Hospital-Compare/Hospital-Readmission-Rates/92ps-fthr/data https://data.medicare.gov/Hospital-Compare/Hospital-Readmiss...

Fomite 8y ago
"Tiny hospitals" have low readmission rates, because the complicated patients with lots of readmissions get transferred to tertiary care centers. This also came up in things like Consumer Reports and their analysis of hospital acquired infection data. Small community hospitals with uncomplicated patients end up looking really good.

wolfi1 8y ago

when such terms as "unsupervised learning" are coined I doubt that there was something achieved, common sense wouldn't have achieved, too. As an AI-expert once said to me: unsupervised learning does not work