5 ms·

You found a singular NEJM paper by the celebrated "checklist champion" Atul Gawande for a method that applies only to one specialty, surgery. How can you be sur

by pak 11y ago

You found a singular NEJM paper by the celebrated "checklist champion" Atul Gawande for a method that applies only to one specialty, surgery. How can you be sure that this generalizes to all medicine? Given that doctors in most other fields spend a ridiculous (by some metrics, 7:1) ratio of time doing paperwork or data entry vs. seeing the patient [1], I would suggest that there is serious opportunity cost in adding more self-measurements and checklists to most doctors' workload. Surgeons are likely an outlier that perhaps benefit most from being regressed to the mean. [1]: http://well.blogs.nytimes.com/2013/05/30/for-new-doctors-8-minutes-per-patient/ http://well.blogs.nytimes.com/2013/05/30/for-new-doctors-8-m...

darawk 11y ago
I can be sure that it generalizes to all medicine because it's obvious. It generalizes to all fields. It's why we write unit tests. Nobody likes writing tests, but we do it anyway because they work. Like, really, really well. The idea that doctors would object to being asked to do the same when people's lives hang in the balance is crazy to me. Now, it's certainly possible they're doing a whole bunch of useless paperwork. And it's certainly possible that there is too much in some areas - but that isn't a rejection of metrics and objective criteria. It's a rejection of the particular implementations of those things in particular places. EDIT: To be clear, I think that what is actually needed is an overhaul of the tech. used to collect these metrics and fill out this paperwork. It's absolutely insane to me that doctors still use paper at all, or that they have to enter the same data multiple times in multiple places, constantly. That needs to be fixed. And they need to start collecting 100x more data on their performance with 100x less friction of collection, by upgrading their technology in this regard from what was available in 1970.
- pak 11y ago
  I can totally understand your sentiment. I would just note that writing a unit test for a class and designing a metric that unambiguously measures good clinical care are on very different levels of difficulty. In our research group, we've tried to do some of the latter regarding infection control, and there is definitely a tradeoff between making metrics explainable in plain language (allowing mental buy-in by healthcare workers) and measuring attributable differences in performance, which usually requires a heap of tricky statistical corrections that few people can grok. It would be a very interesting future where healthcare workers have the reams of stats that professional athletes do, and where everybody in the space is literate enough to intuitively understand each number and its caveats, like hardcore sports fans. For example, an ICU nurse is bound to encounter more hospital-acquired infections (HAIs) per week than a nurse in pre-op assessment, and so averaged cases of HAI per primary unit is the proper context for that stat, much like you'd weigh recent rushing yards for an NFL linebacker against the strength of the defenses faced. Then, managers could optimize their teams to each person's strength or weakness and/or supply training in the correct areas, much like coaches in sports do. This would be a sea change in the medical culture, though, and it would encounter staunch resistance, because some of the metrics already being pushed on doctors (# pts/day) arguably incentivize worse care. To your last point, part of my research now is analyzing electronic medical record (EMR) data, and coming from any other field you'd be shocked at how messy it can be. In a way, this is to be expected, because the "vocabularies" for medical data are huge, and even widely used ontologies have bizarre properties [1]. I really hope that we can solve the data collection problems with better technology, but as of now, the incentives for EMR vendors to overhaul the tech is pretty low, now that most US hospitals have picked a system and those vendors have everything to lose and little to gain from overhauling their user interface. [1]: http://www.healthcaredive.com/news/the-16-most-absurd-icd-10-codes/285737/ http://www.healthcaredive.com/news/the-16-most-absurd-icd-10...

by pak 11y ago

darawk 11y ago

I can be sure that it generalizes to all medicine because it's obvious. It generalizes to all fields. It's why we write unit tests. Nobody likes writing tests, but we do it anyway because they work. Like, really, really well. The idea that doctors would object to being asked to do the same when people's lives hang in the balance is crazy to me. Now, it's certainly possible they're doing a whole bunch of useless paperwork. And it's certainly possible that there is too much in some areas - but that isn't a rejection of metrics and objective criteria. It's a rejection of the particular implementations of those things in particular places. EDIT: To be clear, I think that what is actually needed is an overhaul of the tech. used to collect these metrics and fill out this paperwork. It's absolutely insane to me that doctors still use paper at all, or that they have to enter the same data multiple times in multiple places, constantly. That needs to be fixed. And they need to start collecting 100x more data on their performance with 100x less friction of collection, by upgrading their technology in this regard from what was available in 1970.

pak 11y ago
I can totally understand your sentiment. I would just note that writing a unit test for a class and designing a metric that unambiguously measures good clinical care are on very different levels of difficulty. In our research group, we've tried to do some of the latter regarding infection control, and there is definitely a tradeoff between making metrics explainable in plain language (allowing mental buy-in by healthcare workers) and measuring attributable differences in performance, which usually requires a heap of tricky statistical corrections that few people can grok. It would be a very interesting future where healthcare workers have the reams of stats that professional athletes do, and where everybody in the space is literate enough to intuitively understand each number and its caveats, like hardcore sports fans. For example, an ICU nurse is bound to encounter more hospital-acquired infections (HAIs) per week than a nurse in pre-op assessment, and so averaged cases of HAI per primary unit is the proper context for that stat, much like you'd weigh recent rushing yards for an NFL linebacker against the strength of the defenses faced. Then, managers could optimize their teams to each person's strength or weakness and/or supply training in the correct areas, much like coaches in sports do. This would be a sea change in the medical culture, though, and it would encounter staunch resistance, because some of the metrics already being pushed on doctors (# pts/day) arguably incentivize worse care. To your last point, part of my research now is analyzing electronic medical record (EMR) data, and coming from any other field you'd be shocked at how messy it can be. In a way, this is to be expected, because the "vocabularies" for medical data are huge, and even widely used ontologies have bizarre properties [1]. I really hope that we can solve the data collection problems with better technology, but as of now, the incentives for EMR vendors to overhaul the tech is pretty low, now that most US hospitals have picked a system and those vendors have everything to lose and little to gain from overhauling their user interface. [1]: http://www.healthcaredive.com/news/the-16-most-absurd-icd-10-codes/285737/ http://www.healthcaredive.com/news/the-16-most-absurd-icd-10...