14 ms·
What happens when patients find out how good their doctors really are? (2004)
- conorh 13y agoI hope that more and more data becomes available around healthcare and outcomes! Medicare recently produced this data http://www.nytimes.com/interactive/2014/04/09/health/medicare-doctor-database.html http://www.nytimes.com/interactive/2014/04/09/health/medicar... and the center that my wife works at used it to to produce this graph http://blog.parathyroid.com/parathyroid-surgery-medicare/ http://blog.parathyroid.com/parathyroid-surgery-medicare/. They are quite a bit better at this particular operation than anywhere else in the US, not surprising given how many more they do, however it is often difficult for patients to know or understand that - who should they trust? More data please! Interestingly this operation is one that Atul Gawande (who wrote this article in 2004) specializes in too.
- angersock 13y agoIf you're interested in referral patterns: http://omni.docgraph.org/ http://omni.docgraph.org/
- arn 13y agoGreat article. Might be hard to get through, so I'll put out this quote which I enjoyed and compelled me to read the rest of it. "Matthews had started a cystic-fibrosis treatment program as a young pulmonary specialist at Babies and Children’s Hospital, in Cleveland, in 1957, and within a few years was claiming to have an annual mortality rate that was less than two per cent. To anyone treating CF at the time, it was a preposterous assertion. National mortality rates for the disease were estimated to be higher than twenty per cent a year, and the average patient died by the age of three. Yet here was Matthews saying that he and his colleagues could stop the disease from doing serious harm for years. “How long [our patients] will live remains to be seen, but I expect most of them to come to my funeral,” he told one conference of physicians. In 1964, the Cystic Fibrosis Foundation gave a University of Minnesota pediatrician named Warren Warwick a budget of ten thousand dollars to collect reports on every patient treated at the thirty-one CF centers in the United States that year—data that would test Matthews’s claim. Several months later, he had the results: the median estimated age at death for patients in Matthews’s center was twenty-one years, seven times the age of patients treated elsewhere. He had not had a single death among patients younger than six in at least five years."
- yesiamyourdad 13y agoWhat grabbed me: "In this short speech was the core of Warwick’s world view. He believed that excellence came from seeing, on a daily basis, the difference between being 99.5-per-cent successful and being 99.95-per-cent successful."
- hsitz 13y agoI took notice of that passage also. But the main reason the 99.5 to 99.95 percentages is meaningful is that they were measures of a daily risk. So it quickly compounds into a meaningful percentages; the 99.95% rate meant an 83% chance patient would stay well this year, the 99.5% rate meant only a 16% chance of staying well for this year. Many of the medical percentages that we see quoted in media are derived using a lifetime risk basis, _not_ a daily one. On a lifetime basis the difference between 99.95% and 99.5% _is_ negligible. I wonder how many people reading this article are aware of the difference.
- bdcravens 13y agoI have Cystic Fibrosis. I don't think it's fair to assume that the bell curve associated with treatment results is fair to attribute to the quality of care. Factors such as climate and pollution can significantly contribute to health, as can socio-economic and average distance patients travel, as well as access to insurance. For instance, I'm from North Texas; I went to college in Oklahoma. Back then (1996) Texas had a type of insurance for children with CF (it actually extended to age 21) unavailable in Oklahoma. I ended up losing care, going off of my meds, and by the time I was back in Texas and had the income to support my needs (I came from a very poor family) I had lost 30 pounds (down to 98lbs) and had lost 25% of my lung capacity. Was this attributable to the clinic in Dallas? Of course not. (FWIW, I'm up to 155lbs these days and my lung capacity is probably in the upper 5-10% of all CF patients)
- rcthompson 13y agoOf course there are other effects that go into that bell curve (I'd be more worried about doctor-to-patient ratios and time per patient), but I think the article does an excellent job of highlighting the differences in care.
- bdcravens 13y agoVery true. It's important to note that the majority of a (well maintained) CF patient's health is the result of the the breathing treatments and enzyme treatments; I could probably not see a doctor for years if I had my meds. (Probably true of many chronic conditions I'm sure, but speaking to my experience only)
- abdulhaq 13y agoI'm sure you know this but a good physician will closely track your lung function and on a dip will get a sputum test done and then get you on the right antibiotics asap. That is a very important part of CF treatment that does need regular trips to the clinic. BTW I'm glad you're doing so well :-)
- 13y ago
- rcthompson 13y agoWhat stuck out for me was the difference in attitude between the two example centers. It's "67% is still pretty good, so keep it up" vs "90% is a big decrease from 105%; this is a failure and we need to make adjustments." In articular, the former type of attitude is certainly the norm from my experience (with medical care in general, I have no experience with CF).
- timbre 13y agoThis idea applies to more than medicine. A boss once told me, speaking about a piece of work I thought was pretty good, "If it could be better, it's wrong." It stuck with me, and I've found that everyone I meet who is exceptional at some skill has this attitude.
- DannoHung 13y agoYeah, that's sorta stuck with me. I know a lot of people who say that the perfect is the enemy of the good. Well, buddy, that just means that if you have a good solution, it's stopping you from getting a perfect solution.
- anigbrowl 13y agoHmm, no. 'The perfect is the enemy of the good' is one way to sum up the law of diminishing returns, and when you're operating within fixed constraints that matters a great deal, because inefficiently allocated resources have to come from somewhere. I'm a perfectionist by inclination, but in a team environment perfectionism can manifest as tunnel vision and optimizing for a local maximum.
- watwut 13y agoI met people literally unable to finish project until it was perfect. It was never perfect. They just kept iterating and iterating the first part of the projects and the whole thing was never finished. They were not forced to produce crap by overly active manager, it was not that situation. Manager did not cared for months. Perfect is the enemy of the good refers to the above problem.
- grifpete 13y agoQ.What do you call the guy who passes out last in his class on his med finals? A. Doctor
- jtheory 13y agoQ. What do you call the guy who scored just a teensy bit lower than that? A. Mister. IOW, this joke/insight is only valuable if the standard for passing is set too low. If it's set correctly, then everyone who passed has shown they are prepared to be a competent doctor.
- shabble 13y agoOf course, in the UK/Commonwealth you might offend a surgeon if you call him "Doctor"[1] [1] http://www.bmj.com/content/321/7276/1589 http://www.bmj.com/content/321/7276/1589
- herdrick 13y agoIf competence is, as the article suggests, normally distributed, then there's no natural level at which to draw the cutoff line. But anyway, iirc, studies show no correlation between outcomes and class rank (nor selectiveness of medical school), at least for surgeons.
- trustfundbaby 13y agoOstensibly an article about medicine but really more about how to achieve true excellence at what you do ... definitely a long read but well worth the time
- jsudhams 13y agoThe same question came to my mind and I asked a doctor "Sir, what happens to the students in MBBS who just pass taking a border mark?" The doctor said you should still be ok because passing score itself is like 60% + lot of practice involved. I did not bother to check the pass mark required for the course but now-a-days I go to doctor who have at least 10+ years of experience.
- lmm 13y ago> now-a-days I go to doctor who have at least 10+ years of experience. That's not very Kantian. How are less-experienced doctors supposed to get experience?
- jsudhams 13y agoWith some else :) Keeping the joke aside. I would want them to assist senior doctor for some time and personally i always use second opinion. BTW, if you do not you can get anti-biotic off the shelf and most of the doctors will give you in first visit.
- deleted 13y ago[deleted]
- altrus 13y agoSignificantly, this article neglects to consider how sample size (and demographics) might affect the measurement of hospital treatment quality. tl;dr: If you don't adjust for the sample size, what may appear to be the best, or worse, hospital, may look like that only because of how they have (un)successfully treated a single patient, and thereby yield an unrealistic estimate of patient quality (ie; 100% cure rate of a single patient, which may not be reflective of the actual caliber of doctors). Further reading, which is strongly recommended if you want to learn more; http://nsmn1.uh.edu/dgraur/niv/TheMostDangerousEquation.pdf http://nsmn1.uh.edu/dgraur/niv/TheMostDangerousEquation.pdf
- danso 13y agoSorry...but I don't get what you're postulating...that the doctors and hospitals whose reputations are at stake...nevermind the lives of their patients who are affected by policy decisions...would be totally ignorant of that basic statistical caveat? That it's possible that the top-of-the-line hospital just happened to cure a couple of really lucky patients and everyone who has lauded that hospital has never heard of regression analysis? I guess it's possible...but it's probably more likely that the New Yorker is not trying to be a reference on statistical methods. In any case, the statistical caveat you mention is arguably addressed in this catchall-paragraph that briefly describes the problem of quality-of-care statistics: > In recent years, there have been numerous efforts to measure how various hospitals and doctors perform. No one has found the task easy. One difficulty has been figuring out what to measure. For six years, from 1986 to 1992, the federal government released an annual report that came to be known as the Death List, which ranked all the hospitals in the country by their death rate for elderly and disabled patients on Medicare. The spread was alarmingly wide, and the Death List made headlines the first year it came out. But the rankings proved to be almost useless. Death among the elderly or disabled mostly has to do with how old or sick they are to begin with, and the statisticians could never quite work out how to apportion blame between nature and doctors. Volatility in the numbers was one sign of the trouble. Hospitals’ rankings varied widely from one year to the next based on a handful of random deaths. It was unclear what kind of changes would improve their performance (other than sending their sickest patients to other hospitals). Pretty soon the public simply ignored the rankings. Even with younger patients, death rates are a poor metric for how doctors do. After all, very few young patients die, and when they do it’s rarely a surprise; most already have metastatic cancer or horrendous injuries or the like. What one really wants to know is how we perform in typical circumstances. After I’ve done an appendectomy, how long does it take for my patients to fully recover? After I’ve taken out a thyroid cancer, how often do my patients have serious avoidable complications? How do my results compare with those of other surgeons? (the author himself is a surgeon and has written a lot about the problems of reliably measuring quality of care performance)
- aidenn0 13y agoDoesn't the central limit theorum imply that most sane distributions of quality of care (including all doctors are equally good) will result in a bell curve? There are tests you can do to tell if the variance between care centers exceed that you would expect from chance, but the mere observation of a bell curve seems completely uninteresting.
- hessenwolf 13y agoSay each doctor's patients had a common constant death rate for that doctor, not a crazy assumption, and those constant death rates across doctors followed a gamma distribution (not hard, because I could fit my ass with a gamma), then the distribution of each doctor's patients' life expectancies would be exponential and the distribution of life expectancies of the whole group would be Pareto. Exponential is completely positively skewed and thin-tailed, and Pareto is roughly the same shape but very fat-tailed.
- droopyEyelids 13y agoGood article, but I feel Atul Gawande mischaracterized "evidence-based practice" in a way that does our whole planet a disservice when he poo-poohed it to lionize Warwick. Evidence based practice isn't really new, it's been a thing since '92. And while it is about using studies to make treatment decisions, it's not about forbidding doctors from thinking for themselves. But what really got my goat was the fact that evidence-based medicine is the new thing, and it's an essential step in the evolution of medicine, and more people need to understand it. Evidence based medicine is about applying statistics to determine when treatments cause more harm than good. With modern medicine's advances in imaging techniques, we're hitting all sorts of new and dangerous problems where the benign abnormalities we all accumulate over our lives are treated at great expense, inconvenience and loss of health.
- ars 13y agoI don't think he was "poo-poohing" it exactly. If everyone only follows evidence based medicine there will never be any improvement because there will never be any changes. So someone has to be the person who ignores it, who tries new things. And that someone, in this field, is Warwick. The evidence part comes in when that person also tracks their results! i.e. makes it a study, and Warsick certainly seems to do so.
- 6cxs2hd6 13y agoYes I got the impression his attitude about evidence-based medicine is similar to that saying about accounting -- that "accounting is the art of looking in the rear view mirror and dragging your ass into the future". i.e. Accounting isn't worthless. It's necessary, but not sufficient.
- PaulJulius 13y agoPatients with CF at Fairview got the same things that patients everywhere did—some nebulized treatments to loosen secretions and unclog passageways (a kind of mist tent in a mouth pipe), antibiotics, and a good thumping on their chests every day. Yet, somehow, everything he did was different. I found this quote especially interesting. I've often found this to be true, that you look over at someone who's much better, and on one hand they're just doing the same things you're doing, but somehow they're just on a different level. It's very difficult to distinguish exactly how someone achieves better results, that is, until it's been explained. Here, we see this clearly in how the level of lung function is perceived. Both centers provide the same treatment, but at the Cincinnati center lung levels around 70% are "okay", but at Minnesota, any drop is unacceptable, even if the patient is already at above-normal levels. It sort of reminds me of the "Don't deal with it, fix it" article that was also posted recently. You'll be coasting around at a certain stable level of engagement until one day you see something you've never considered before and all of a sudden you see a whole new world of possibilities.
- graeham 13y agoMedicine is like any other profession - it shouldn't be surprising that some doctors are better than others, like some lawyers/engineers/teachers are better than others. There are a lot of challenges with healthcare metrics. Medical problems are more complex, arising from more factors than practically any other problem. Most other sciences can be resolved by theory or relatively convergent empirical data. Medicine is empirical, but the scatter is huge. Data is not tracked well. When it does exist, it is in formats that are not well conducive to analysis and sharing. This seems to be slowly (and expensively) improving. Pressure for privacy adds to inertia for innovations that could use medical data for improved outcomes. With the title of the article, I was hoping there would be some more discussion about implications of some 'rating system' for doctors. It would probably superficially be a good idea, but such a system would obviously result in some level of gaming. Does a doctor in a rated system only take easy cases to keep a good track record? Are hard cases given more weighting? Who/ how is such a weighting decided? Such a system could offer a results-based compensation scheme for doctors, which is lacking from most publicly funded systems.
- derekp7 13y agoWhat about the equiv. of code reviews? For surgeons, take videos of each surgery, and have them rated by an impartial review panel. I can't find the link now, but I recently saw a segment posted on Youtube that compared a surgeon with good technique, to one with poor technique, and it correlated with the overall outcome of the patients.
- FLUX-YOU 13y agoI remember this, but I also can't find it. Part of it involved maintaining proper view of the field through laparoscopes, minimizing wasted movement, and avoiding unnecessary tissue damage. That last one has the most direct effect on outcomes, while the former two can lead to the third. The lower ranked surgeons would have the scope jostling around and the laparoscopic instruments would contact nearby tissues, while the higher ranked surgeons just seemed to have more elegance in his skill; a laser-precision focus to borrow from this thread and an intention and purpose for every movement. I had a similar impression watching that video on surgery technique to watching high level Starcraft player replays versus some afternoon ladder replays by mid-level players. When placed side-by-side, the differences were fairly obvious even if you've never done the act yourself.
- SeanDav 13y agoThis reminds me of a true story I once read about the best car salesman in the country at the time. Single handedly he was selling more than some not-so-small entire dealerships. The difference was his incredible focus and approach to even the tiniest detail that could help a sale and willingness to try new things. One thing that stands out in my memory about this was the fact the he made sure he always had 3 (!) pens for the customer to sign the sales agreements. Just in case 1 pen stopped working and the backup pen malfunctioned. He brought this sort of laser focus to absolutely every aspect of the business. How many salesmen would have a spare pen, already in place, let alone 2 spares. How many would even have thought about it and what the right number might be? Focus, attention to detail and continuous innovation.
- Terr_ 13y agoUnfortunately, there's also a cargo-cult aspect to this kind of thing, when managers start saying: "Everyone is now required to have three pens at their station at all times." I'm in a company where the suit-wearing side says "we need innovation" to the tech side, but they don't actually want a discussion about what they see as opportunities or directions. They just want to invoke it by rote ritual, or order it as if we kept some on shelves.
- deleted 13y ago[deleted]
- Terretta 13y agoMoney quote, applicable to hacking in general: “We are used to thinking that a doctor’s ability depends mainly on science and skill. The lesson from Minneapolis is that these may be the easiest parts of care. Even doctors with great knowledge and technical skill can have mediocre results; more nebulous factors like aggressiveness and consistency and ingenuity can matter enormously... What the best may have, above all, is a capacity to learn and adapt—and to do so faster than everyone else.”
- rsanek 13y agoReally good takeaway near the end of the article: "What the best may have, above all, is a capacity to learn and adapt—and to do so faster than everyone else."