17 ms·

Heart surgeons refuse difficult operations to avoid poor mortality ratings

434 points by prostoalex 9y ago

rdiddly 9y ago
Goodhart's Law again!
aantix 9y ago
The subjective is always where there is value. I wish there were surgeons that would accept these challenges and colleagues would objectively assign a degree of difficulty score. And a site that aggregated such scores. I really don’t care how “friendly” a doctor is - did the get the diagnosis right? Did they improve longevity? Did they improve quality of life? Not easy to score but what everyone wants to know.
- slap_shot 9y ago
  Not sure why this is being downvoted. It’s a legitimate request. I worked for a novel medical device company and we had a lot of surgeons on staff. While not a product we’d ever build, there was always discussion amongst the doctors of how to better grade physicians and specifically surgeons. The concern is was that a lot, and I mean a LOT, of surgeons were bad at their job. They were always proposing different systems to evaluate surgeons, very close to what you described.
  - ItsMe000001 9y ago
    > ... and colleagues would objectively assign a degree of difficulty score. He asks for a level of "objectivity" that you just can't get. What do you think medicine is? Like repairing toasters? "Why don't you simply assign some objective numbers" - it's so easy! No it is not, unless you only count a set of basic and routine procedures. Even things like a broken bone vary greatly - and then add the additional variance of the people having those broken bones. You can sort of get some objectivity when you do statistics of lots and lots of patients in many places over long periods - but the problem OP wants is to be able to make statements about individuals (doctors). Being able to say something about a population is very different from saying something about an individual within the population. "Statistics" about a single doctor are and will be full of randomness. Besides, if they did what I quoted from OPs statement above what would probably happen is that everybody declares everything as "very difficult" (or just below) expecting colleagues to do the same for them. In Germany private insurance has a multiplier on the amounts they can charge for something. Guess what: Everything is now declared as "very difficult" (with some short fuzzy explanation), multiplier 2.3. Almost nothing is charged at the "normal difficulty" rate. > Did they improve longevity? Did they improve quality of life? Given that each doctor is highly specialized, and that on the other hand your GP, the one who treats you long-term, can't actually do much when there is anything serious, those numbers are mostly out of the control of individual doctors. You would have to get such numbers system-wide, not per doctor. Medicine is supplied to patients throughout their life (incl. late life) by a network, not by an individual. There also is a huge impact of ones own life choices (e.g. obesity, smoking), and no, you can't just correct for them - not individually (for populations we can). All those numbers would look terrible for doctors who treat more of the worse cases. It would lead exactly to what the article is about. I'm not arguing against trying to measure surgeon outcomes - I'm against trying to use those measures for things they are not good for.
    - aantix 9y ago
      The feedback would have to be anonymous. It’s the only way to get honest scoring.
- carbocation 9y ago
  What you're describing is commonly referred to as "propensity weighting." It is a good first step, though most of us think it still doesn't do a sufficient job of accounting for the risk of difficult cases.
- sky_rw 9y ago
  Does the NHS provide enough free market choice that anybody could take advantage of a score even if they wanted to?
  - dannyw 9y ago
    If you have private insurance, you can choose your own surgeon.
  - ealexhudson 9y ago
    You can choose where to get treatment and who treats you; the main issue is whether you get the timeline you'd prefer. Generally, patients will choose a clinical center (to the extent that some hospitals have stopped offering specific treatments, because their stats were never good enough - e.g you need to do a certain number of children's heart ops per year to be good enough to get good stats). If you have the option to go private, either through your insurance or the NHS, then obviously there's another layer of choice. Most of the private surgeons are also NHS, anyway.
  - SmellyGeekBoy 9y ago
    Yes, my father is currently undergoing treatment for cancer (involving multiple surgeries) and had issues with a surgeon. He was given the option to choose a different surgeon or even transfer to another hospital entirely.
  - DanBC 9y ago
    It provides some. "Choose and Book" was introduced in 2004, and converted into e-referral in 2014, and it means you get a choice for most elective surgery. https://www.nhs.uk/NHSEngland/appointment-booking/Pages/about-the-referral-system.aspx https://www.nhs.uk/NHSEngland/appointment-booking/Pages/abou... https://www.england.nhs.uk/2014/05/choose-and-book/ https://www.england.nhs.uk/2014/05/choose-and-book/ Most people don't have enough information to make a sensible choice; and they don't understand the information they've got.
firstplacelast 9y ago
Check out the comments over on /r/medicine and you will likely be horrified. It is career over everything. There was even a thread this week about how you should NEVER correct someone above you (med students should never correct a resident and a resident should never correct an attending). That mentality is terrible for everyone and every thing, except the physicians' careers. I don't know if we need to select for better people when admitting them to medical school or if the culture is just completely fucked.
- theatrus2 9y ago
  Such practices were pervasive in other industries too, including air crews. Thankfully, this has been changing, and shown marked improvement in some cases.
- Spooky23 9y ago
  It’s a tough profession populated with type-a people, paid by insurance companies and beholden to malpractice insurers as well. As with anything in modern society, bean counters rule and usually drive awful decision making.
- bradlys 9y ago
  If you think about what it takes to get into med school then it seems pretty natural that you're going to get hyper competitive surgeons in the end. It starts before med school. You get assigned 10 unobtainable books that only the library has. Finals week comes around, all the books are missing even though they have to stay in the library. Turns out your classmates hid them inside the library so they could have a competitive advantage and get a higher grade on the curve. You already devoted probably half your life to becoming a surgeon, why would you screw it up and lose so much ground so easily?
  - carbocation 9y ago
    What decade are you describing? In my medical school, all of the required material aside from our anatomy textbook was given to us as a PDF or a printout.
sebazzz 9y ago
Would it be possible to determine a "risk" factor and count that in the mortality ratings, or is this too simple?
- MarkMc 9y ago
  There is probably already some adjustment for risk - anyone can see that a smoker in his 60s will have higher mortality than a non-smoker in his 30s. The problem might be that the statistician doing the risk adjustment doesn't have 'skin in the game', so his/her assessment of risk is worse than the surgeon's. (Although if that were the case I would expect the statistician to be just as likely to overestimate the risk than underestimate it, yet I never hear about surgeons favouring the difficult cases because they think they can improve on the expected mortality rate in such cases).
carbocation 9y ago
The advent of public reporting of outcomes after intervention for myocardial infarction (heart attack) led to an increase in mortality in Massachusetts and New York.[1] 1 = https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4368858/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4368858/
- fricat1ve 9y ago
  A better way to make this claim would be to plot the mortality rates for the states over time and show that the rates in those places requiring reporting begin to diverge as the new policy is implemented.
  - carbocation 9y ago
    They effectively did that: they compared public reporting states with other geographically adjacent states.
    - fricat1ve 9y ago
      There's nothing like Boston or NYC in the control states. If the effect is as big as they seem to think--a 21% increase in mortality--that should be visible on a plot of deaths over time.
tgraham 9y ago
66% of 115 specialists polled did not refuse a difficult operation. 33% thought about how it would impact their statistics, or put more generously, thought hard about the probability of the patient surviving presumably quite serious surgery. Yes mortality is a blunt statistic for a surgeon, but perhaps giving people pause for thought about the odds of success is no bad thing. Contra point: would you rather not know how many cases your surgeon has done that year, and the issues they have had?
- maxerickson 9y ago
  A large portion of the people undergoing heart surgeries still face a high risk of death if they don't have the surgery (with the surgery bringing the chance of a longer, more comfortable life). Here's why your hot take isn't the best take: “About 30 percent of them said they had turned patients down for surgery even when they knew full well that surgery was in their best interest.”
  - TheRealPomax 9y ago
    Here's the converse of that, though: why do we think it's okay to expect a team of doctors and nurses to be okay with almost assuredly killing someone during surgery? They're not robots, "it's their job" is not an acceptable answer. Even if the surgery is in a patient's best interest, if the odds of killing them are all but guaranteed then it's most definitely not a matter of looking at the patient and rationalising it with "they will die otherwise anyway". It's not just about the patient. People who make this argument seem to forget that there's also an entire team of medical professionals that your rationalisation says should be okay with going into a surgery knowing they are almost guaranteed to kill this patient. They have the stats, the stats say "this person will die under the knife", many more lives are affected in this decision than just the patient. So expecting them to just do the surgery instead of going "No. This will kill the patient, I don't want that on me and my team" is very far from an okay attitude towards fellow human beings, and leads to terrible medical practices.
    - maxerickson 9y ago
billfruit 9y ago
On a related vein, do surgical procedures involve any type of quality control, where the work is independently audited? If so I wasn't aware. Quality control is such an important cogwheel of processes in many other industries, yet I don't hear much talk of it in relation to medical profession.
- Dayshine 9y ago
  Yes, at least in the UK, we have a series of national clinical audit programmes.
- prepend 9y ago
  There are process checklist reviews (http://qualitysafety.bmj.com/content/23/4/299 http://qualitysafety.bmj.com/content/23/4/299) and medical review panels (https://mrp.ky.gov/Pages/index.aspx https://mrp.ky.gov/Pages/index.aspx).
  - deleted 9y ago
    [deleted]
- fricat1ve 9y ago
  In the US, if the surgery is in hospital: - Well-regarded hospitals will vet surgeons before granting privileges. - Average hospitals give out privileges fairly easily if there are no actions against a person's license. - There is a "collegial" review of big screw-ups that carry major reputational risk. - As with any fee-for-service firm, "rainmakers" are highly sought after and get away with more. - Privileges are difficult to take away once granted. (Have stronger legal protections than academic tenure in some states.) If the surgery is in an ambulatory surgery center or doctor's office: Basically anything goes. Ice-pick lobotomies, tonsillectomy mills, boob job factories in strip malls....all have happened in recent US history. About as well-regulated as traveling carnivals.
  - billfruit 9y ago
    I was thinking something more akin to routine inspection program carried out by independent personnnel applicable to all procedures, not just the reviews that are necessitated due to screw ups.
drewg123 9y ago
It was not heart surgery, but after being diagnosed with lung cancer, my mother had one of her lungs removed by a surgeon who assured us he'd be able to get all the cancer. She had a miserable, painful recovery that I would not wish on anybody. Within a week or two of her recovering to the point where she could walk short distances, the cancer was found in her other lung and went on to kill her within a few more months. She may have gained several months of life from this operation, but it was a painful few months without dignity. It was 20 years ago, so my memory is fuzzy, but I don't remember ever being presented with anything but certainty from the surgeon that he could cure her. I think that patients and surgeons need to have a better idea of the realistic chance of success from a major surgery in order to make an informed decision. Perhaps if surgeons at least have to worry about their own numbers, they might hesitate before attempting cases like my mother's.
- endymi0n 9y ago
  How doctors die (2011): https://news.ycombinator.com/item?id=13122408 https://news.ycombinator.com/item?id=13122408
- unpopular42 9y ago
  That surgeon may have made a mistake, but you're suggesting to inject (more) unrelated incentives into this kind of decisions and expect them to get better? I'm sorry, but this sounds completely idiotic.
- metaobject 9y ago
  I guess I'm just shocked that the surgeon could offer that level of assurance about something like that.
  - astura 9y ago
    Unbelievable! The five year survival rate for lung cancer is only 18%! Even when the cancer is localized the five year survival rate is only 55%![1] It's irresponsible for the doctor to not give a reasonable prognosis! [1]http://www.lung.org/lung-health-and-diseases/lung-disease-lookup/lung-cancer/resource-library/lung-cancer-fact-sheet.html http://www.lung.org/lung-health-and-diseases/lung-disease-lo...
samfisher83 9y ago
Sports athletes do this. For example players won't attempt end of the quarter half court shots since that will lower their percentages, or QBs taking sacks to not lower their QB ratings. What they do in sports is keep advanced stats. Instead of just giving a pure mortality rating have mortality rating, by surgery type.
- deleted 9y ago
  [deleted]
- make3 9y ago
  Then they just dodge the hard cases of each specific type of surgery. It's probably still an improvement though on just avoiding a surgery type altogether.
  - killjoywashere 9y ago
    Then you track the deferral rates.
    - jessriedel 9y ago
      ...which punishes doctors for serving high-risk population.
    - TheCoelacanth 9y ago
      Then they start operating on people who don't really need it.
- skookumchuck 9y ago
  One could break out the quarter half court shots as a separate statistic.
  - paulcole 9y ago
    At this point it’s very likely all teams do. The stats they’re using are nothing like what was printed on the back of basketball cards when we were kids.
trackofalljades 9y ago
Sounds a lot like police work in Japan (they love their stats).
cs702 9y ago
It's Goodhart's Law in action, this time with deadly consequences: "When a measure becomes a target, it ceases to be a good measure."[a] [a] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- batbomb 9y ago
  Ironically applied to not-good hearts
  - truculation 9y ago
    In another sense, 'Goodhart' seems like a splendid cosmic coincidence. Instead of obsessing over targets, try to perceive the whole system. And who can do that? Good-hearted individuals.
  - mjfl 9y ago
    "These doctors are vile dogs. If I was a physician I would take all the hard patients, heal them all, and be a hero of the people." Yeah, if you were given one operation where the patient has a 100% chance of dying without it, but 80% of you killing them if you operate + 20% chance of them living after, maybe you'd do this once and get lucky. But try doing this twice, you have a a 96% chance of killing a patient, three times makes a 99.2% chance of killing at least a single patient. This probability quickly approaches 100%. If you kill a patient you get dragged into court, branded as a murderer by the opposing lawyer, and your career which you have spent your youth, your 20s and a half a million dollars training for is down the drain. Taking risky cases, if that's not your niche, is is guaranteed to have you out of a job and in debt from legal fees. To suggest that "bad hearts" are involved in this necessary calculation, is incredibly naive.
    - smnrchrds 9y ago
      I am pretty sure that the parent was referring to the fact that this article is about heart surgeries. Bad heart is used in the same manner as a bad knee or a bad tooth. It refers to the patient not the doctor. But aside from that, I completely disagree with your comment. The approach you are defending, even if legal, is completely unethical and immoral. It should be something we chastise, not condone.
deleted 9y ago
[deleted]
JumpCrisscross 9y ago
If you’re a patient, wouldn’t this still mean starting with the lowest-mortality surgeon? If they take your case, you’re least likely to die. If not, you have a better appraisal of your odds.
- KMag 9y ago
  That strategy may very well end up skipping (well, delaying selection) of a surgeon that routinely takes much harder cases than yours. I don't know enough about heart surgery to know if practice at more difficult procedures has any carryover to less difficult procedures, but it's conceivable . Your strategy might make sense, but it's not obviously optimal.
- stordoff 9y ago
  I'm not so sure. That might mean going with a surgeon where you are at the top end of their "risk profile", rather than a more skilled surgeon who has higher mortality rates due to routinely taking more difficult cases. I'm not sure you can generalise the mortality rate of all patients to your specific chance of death (a more skilled surgeon would likely be better for you, c.p.).
amelius 9y ago
Why don't we keep track of refusals as well?
- B1FF_PSUVM 9y ago
  Not a bad idea, per se, but then which is better - the doctor with 0% refusals and 20% deaths, or the one with 20% refusals and 10% deaths?
  - wellboy 9y ago
    You compare it to the average death rate of a surgery.
  - dredmorbius 9y ago
    Include the outcomes of the refused cases. Note that this also works to improve stats on docs who choose hard cases.
agumonkey 9y ago
it's been said a few times on /r/medicine, surgeons have their lawyer in mind when operating, they're skilled, but they first avoid anything that could damage their own life irreversably. Of course to an extent this goal aligns with the patient, but not always, and a bit sad on the ideal side.
- kankroc 9y ago
  Can someone really sue their surgeon in the US?
  - agumonkey 9y ago
    I can't recall what countries these guys were.
  - KMag 9y ago
    Absolutely. You can sue for pretty much anything. Winning is another story, but doctor's don't generally make sympathetic defendants. Juries know they have money and that they pay huge insurance fees to cover these cases. Of course, after losing a big case where the insurance company pays out millions of dollars, they likely become uninsurable and unemployable. But hey, the money is good until you hit the anti-lottery. My father was an anesthesiologist (who retired in good standing), and I remember him telling me a case he read about a Caesarean section. The surgeon was using an electrocautery pen to sear closed the ends of blood vessels. The surgeon set down the pen on the surgical cart, lifted the baby out, set the baby on the cart, and the baby's heel touched the pen. Now the baby has a small scar on the bottom of his/her heel for life. The parents sued the surgeon, the cart nurse overseeing the cart, and the anesthesiologist was close to the surgical cart. All 3 defendants settled out of court, since babies almost always win jury cases against rich doctors, regardless of merit. The litigant's lawyer would almost certainly bring up a below median survival rate as evidence of a pattern of gross incompetence. Also, of course, in the U.S., I believe only governments are immune to lawsuits (sovereign immunity) without their consent. The courts have consistently ruled that many rights, including the right to sue, cannot be legally waived by contract. In a similar way, if you sell yourself into slavery by entering into a contract that waives your right against unlawful detention, that contract cannot be legally enforced. My grandfather was an anesthesiologist. My dad was an anesthesiologist (and did well enough on the MCAT to go to med school after 3 years of undergrad... practiced medicine without an undergrad degree... med school is too competitive today to do such a thing). I was a good student: I graduated from MIT. However, my brother and I saw all of the BS and stress (anesthesiologists have a high suicide rate) and constantly rotating sleep/work shifts and both went into engineering.
trumped 9y ago
Some doctors probably refuse patients that left bad reviews at competing offices...
- Mononokay 9y ago
  "I don't appreciate the service they provide and they're terrible people !!!" "Okay, we won't serve you if you're just going to trash us." "EVIL HOW DARE YOU!"
  - trumped 9y ago
    I meant if you left a bad review somewhere else... Of course someone should not go back where they don't like to go....
    - viraptor 9y ago
      That only works if you have more than one choice available.
      - trumped 9y ago
        The shitty US health/dental insurance system make it hard, but I still would be surprised if most people don't have a choice of more then one doctor...
Numberwang 9y ago
It's sad to hear this when it's totally unnecessary. Hear disease has been a solved problem for decades. Just compare the rates of disease for vegans with the general population.
- brookside 9y ago
  What are the rates?
  - Numberwang 9y ago
    http://www.mercyforanimals.org/american-heart-association-vegan-diet-decreases http://www.mercyforanimals.org/american-heart-association-ve...
    - brookside 9y ago
      > Researchers studied five different diets to see which lowered the risk of heart failure: convenience (fast food and pasta), plant-based, sweets, Southern (sugary beverages and fried foods), and alcohol/salads. What kind of study design is this? It seems like they compared veganism with 4 garbage diets. Seems like all we can extract from this is plant-based is better than eating trash.
- danans 9y ago
  > Just compare the rates of disease for vegans with the general population. "Disease" is such a broad category that it's sort of meaningless here. Maybe veganism is healthier than an animal diet, but the degree of its impact can only be stated only after removing the many other environmental factors that might be contributing to lower disease rates among vegans, including income and location.
  - Numberwang 9y ago
    Yes, lot's of things are always involved. Including air quality etc as I think I saw on HN just the other day. I find it interesting though that the right type of vegan diet will basically make you hypertension and heart attack proof and that it may also reverse heard disease (the only diet to do so). Especially when you look at mortality and costs of healthcare this really need to be more generally recognized.
NTDF9 9y ago
Reminds me of why some devs will not touch critical pieces of code because the next time a bug comes in that module, git log shows they were last editors and will be asked to take a look (even if the bug is unrelated to them)
narrator 9y ago
They need to add a difficulty correlated metric to the statistics. Maybe BMI and age? I am sure IVF clinics wouldn't offer the procedure to anyone over 35 if the rather detailed statistics they are required to report didn't also include the patient's age.
- Lxr 9y ago
  I imagine it’s extremely difficult to come up with a metric that captures everything about the case. BMI, age, tumor type, tumor shape, history of kidney disease, diabetes, where do you stop?
  - MarkMc 9y ago
    You don't need a metric which captures everything about the case - you just need a statistical model whose risk assessment is at least as accurate as the surgeon's assessment. This is not as difficult as it sounds - in Chapter 21 of Thinking Fast and Slow, Kahneman makes a strong case that simple algorithms are very often better than expert clinical judgement.
- zzbzq 9y ago
  Rather than try to figure out some classification criteria, use a predictions market. Have surgeons predict their own chance of success. Preferably, gather multiple second++ opinions for each case. The difficulty of the surgery is based on some combination of the different surgeon's predictions. A surgeon's stats would then be based on his successful and unsuccessful surgeries, weighted against the difficulty. Using this method, not only can you still rank the surgeons based on the successes of their surgeries, but also on the accuracy of their predictions, which will be in larger sample size and also perhaps a better indicator of their mastery of medical knowledge.
Nuzzerino 9y ago
Can we get a (2016) on the thread title?
SilasX 9y ago
We bundle up too many conflicting jobs into the role of the doctor. Rating the difficulty and status of an incoming patient should be completely separate from the job of treating them, and all judgments about doctor effectiveness should be judged relative to those assessments. (Credit: alerted to this misincentive by Yudkowsky’s recent book Inadequate Equilibria.)
adolph 9y ago
This topic was discussed in the last EconTalk [1] podcast featuring the author of a recent book “The Tyranny of Metrics” [2]. I think one of the author’s interest insights is the distinction between metrics for diagnostics and those for incentives. 1. http://www.econtalk.org/archives/2018/04/_what_is_the_ap.html http://www.econtalk.org/archives/2018/04/_what_is_the_ap.htm... 2. https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller-ebook/dp/B076ZWW2MN https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller-ebook/...
- TwoBit 9y ago
  It's what I hate about the legal system. Prosecutors don't care about justice, they care about a record of putting thevmost people away.
skookumchuck 9y ago
One possibility is to offer the surgeons more money for difficult surgeries, and more money for a successful outcome, to balance out the reputational risk to the surgeon.
- dfox 9y ago
  How do you estimate the difficulty in a way that does not involve wrong incentives on the part of the surgeon or whoever is doing the estimation when money is involved?
2skep 9y ago
What would you prefer? A surgeon who expects a poor outcome of your surgery: a)decides to not operate on you? b)decides to operate on you nonetheless? I am in camp a
- calcifer 9y ago
  I'm definitely in camp B. 90% chance of dying on the table beats a 100% chance of dying without the surgery, every single time.
  - jack9 9y ago
    Quality of life matters in most of these marginal cases. If they are going to give you the Konno procedure (scrape extra tissue out of a thickened ventricle), there's tons of things that can go wrong OTHER than death. Death is assured, either way, for everyone.
- stordoff 9y ago
  It depends what the outcome is without the surgery. If I'm dead without it, I want them to operate regardless. If there's only a chance of death, or the surgery is aiming to improve quality of life (rather than prevent [imminent] death), it's more of a judgement call.
emodendroket 9y ago
You would think that they could adjust these figures by patient age and condition.
crusso 9y ago
Ultimately, we need to compare apples to apples by building rating systems that account for the difficulty of treating patients with various, ailments at various ages, with various complications. Think of it in terms of competitive diving, gymnastics, or snowboarding: Your scores depend upon the difficulty of the moves you attempted combined with your performance of those specific moves. The more difficult the move, the more you're compensated for even attempting it when it comes to scoring. The trick then is to prevent physicians from gaming the system by exaggerating the difficulty of the patients they're dealing with. You'd need to use as many objective metrics as possible and possibly some system of having different physicians assess ratings than the ones performing the procedures - maybe in some kind of double-blind fashion to prevent any kind of coordination strategy.
- calcifer 9y ago
  > some system of having different physicians assess ratings than the ones performing the procedures - maybe in some kind of double-blind fashion to prevent any kind of coordination strategy. That would simply result in an unwritten rule where everyone gives everyone else good ratings.
- fricat1ve 9y ago
  The "objective metrics" part is key. Medical records are generally written in a way that weaves a standard narrative of the doctor as either a hero/savior or a valiant defender of a lost cause. Doctors only get investigated when derogatory facts accumulate to an extent that this "standard narrative" collapses. It's not really an objective or scientific standard.
- vannevar 9y ago
  The article itself alludes to a 'EuroSCORE' that purports to rate the risk of mortality beforehand. It seems like the mortality ratings should be risk-adjusted. A double-blind system as you suggest could work like this: First, the rating system would be opt-in on the part of the participating surgeons. It seems likely that patients would prefer rated surgeons, so there would be an incentive to participate but it would not be mandatory. Those who belonged to the rating body would be required to do anonymous risk assessment on some number of cases per month. They wouldn't know who the patients are or who is treating them, they would only get anonymized records. Their risk assessments would be calibrated in the short run against other assessments, and in the long run against outcomes. These calibrated risk assessments would be used to adjust the score of the treating surgeon on the outcome, so that they would be penalized less for mortality in patients with a very high assessed risk.
martyvis 9y ago
In Australia, we have a neurosurgeon Charlie Teo that is well known for operating on "inoperable" brain tumours. While he has a lot of respect in the general community, to many of his surgical peers he is seen as a reckless maverick. I would think his stats on average are bad, but because he will tackle outliers he has won the support of the people that matter - his patients.
wastra 9y ago
The incorrect assumption your readers are making is that heart surgery is a good thing in all cases. America is the leader in futile medical procedures: it may well be a good thing to have fewer risky procedures on patients who were more likely to be harmed. "primum no nocere" may be advanced!
tyingq 9y ago
Metrics do drive behavior. Like, in the IT space, percentage of successful change requests. Encouraged you to power though even if things look bad. Because aborting is a 100% chance of a ding to the metric, while taking the risk is something less. The stakes are lower, of course, but the idea is similar.
paulcole 9y ago
This exact topic was discussed in the 1968 novel Case of Need by Jeffrey Hudson. It was his first book, written while a medical student at Harvard. The novel raised a stir as it was critical of Harvard faculty and in some ways the practice of medicine in general. The book eventually won an Edgar Award. Some may know Jeffrey Hudson better by his given name, Michael Crichton. https://en.m.wikipedia.org/wiki/A_Case_of_Need https://en.m.wikipedia.org/wiki/A_Case_of_Need
tomohawk 9y ago
Relatedly, I've met many people who had a surgery where they used to live and, after moving, other surgeons won't touch them. They have to travel back to where they used to live to get care. Seems really unethical.
probo 9y ago
You cannot compare crude outcomes. You have to adjust for so called "case-mix". For mortality after cardiac surgery, the risk can be calculated using the EuroSCORE II. http://www.euroscore.org/calc.html http://www.euroscore.org/calc.html And even then, often the numbers for individual surgeons are too small to draw conclusions.
qrbLPHiKpiux 9y ago
A colleague of mine was interviewed by a new hospital heart program. He was asked what his mortality rate was. He replied, what do you want it to be?
SteveGerencser 9y ago
I firmly believe that this is why my mother-in-law died. She needed surgery to fix a hernia that basically left her unable to eat at all. Is she did not have the surgery she was going to die. The surgeon stopped the surgery because the 'risk' was too high. 48 hours later she was dead.
- hycaria 9y ago
  You don't die in 48h from not eating. So many elements missing there, this is such a terrible depiction. I hope anyone passing by will have a bit of critical thinking.
imaadrashied 9y ago
This isn't surprising because humans will do what they're incentivized to do (speaking mainly in a professional setting here). Might be effective to add a measure to the transparency that shows something to the effect of, "likelihood to perform life-saving and risky procedures." Therefore a high win-loss with a lower risk threshold would be weighted lower than a so-so win-loss with a higher risk threshold.
ianformanek 9y ago
There might be several decent ways to improve (at least as a next step) to avoid this particular way to game the KPIs: - use an assessed upfront difficulty (if available in measurable form) to adjust for the outcome rating - add additional KPI for % of refused operations (perhaps again adjusted for difficulty)
justonepost 9y ago
Alternatively, you have a good idea of what your mortality rate will be if a surgeon picks you for a case.
txsh 9y ago
Should “NHS” be added to the title? This seems to only apply to British heart surgeons.
mitko 9y ago
Shameless plug - I wrote an essay recently on how any metric becomes a vanity metric http://dimitarsimeonov.com/2018/03/22/the-vanity-metric-paradox http://dimitarsimeonov.com/2018/03/22/the-vanity-metric-para... Some excerpts: - having clear, well defined metrics is the single largest driver of progress within a company - Any metric sufficiently optimized becomes a vanity metric - Building products is not science. What makes a good product is usually dependent on so many factors that are subject to change and evolve. - Over time, a product no longer stands to die if these metrics degrade. - According to EdwardTufte - people and institutions cannot keep their own score Hope someone finds it useful!
randyrand 9y ago
Easily solvable. Have a person whos jobs it is to rank how likely a person is to live. He gets paid based on how accurate he is. Doctors would then be ranked as an offset from this previous mortality prediction.
- Consultant32452 9y ago
  This incentivizes the doctor to maximize the appearance of risk in the medical record.
  - Too 9y ago
    The one doing the estimate should obviously not be the same doctor doing the surgery.
    - Consultant32452 9y ago
      The one doing the estimate will do so based off the records generated by the doctor doing the surgery.
      - dredmorbius 9y ago
        Not for referrals.
        Consultant32452 9y ago
        The only way to get reasonably accurate results with this is to have two surgeons do completely separate diagnostics in a double-blind where neither they nor the patient knows which one is going to be performing the surgery. That seems unworkable at scale.
        dredmorbius 9y ago
        Surgeons don't do diagnostics, as a general rule. Internists, paths, triage, GP, trauma do.
dosycorp 9y ago
That means there's an opportunity / gap in the market. Demand for risky operations by motivated buyers, should lead to some doctors taking on the role / label of risk-takers / explorers. Since they can get the customers other, risk averse doctors are turning away, and pull in rare-treatment seekers on their own.
fedups 9y ago
Econtalk had a good interview this month on this general phenomenon discussing the book "The Tyranny of Metrics" http://www.econtalk.org/archives/2018/04/jerry_muller_on.html http://www.econtalk.org/archives/2018/04/jerry_muller_on.htm...
deleted 9y ago
[deleted]
refurb 9y ago
This is the classic problem where people use averages as a metric, but ignore the fact that variations are not distributed evenly. I remember looking at the economics of a clinic for our product. Someone said "let's just give them a 5% discount as on average they will break-even. That works if you look at on average how much they make, but ignore the fact that every clinic has a different mix of insurance companies and *their own economics are all different."
acjohnson55 9y ago
We chose to have our daughter delivered at a hospital that is renowned for its neonatal ICU. It's obviously a great place to give birth, but if you look on paper, their c-section and other intervention rates are above average. Which makes sense when you understand that they are equipped to handle the highest risk pregnancies. A similar phenomenon occurs in education. I taught high school in Baltimore, where the vast majority of my students were high risk. Baltimore has a few renowned high schools, but by and large, they serve the easiest students.
stevenwoo 9y ago
There was a counterintuitive study of obstetricians that I cannot find right now that showed that in one state over many (10?) years the ones who did the most C-sections were the best doctors - because when researchers looked at the underlying data the realized these doctors treated the patients who had the worst predicted outcomes and beat the national odds - I think they theorized that these doctors had the best ability to choose the optimal method of birth and were not afraid to choose c-section when it was necessary.
- dredmorbius 9y ago
  There's a very strong relationship between experience (number of procs performed) and outcomes. High-volume is almost always beetter. This may of course favour unnecessary ops...)
  - stevenwoo 9y ago
    Yes, I think this was particularly against the grain because mothers are commonly told to look for a doctor/practice who does not resort to a C-section right away, and judge this by the number of C-sections a doctor/practice performs.
    - dredmorbius 9y ago
      Hospital rather than doc rates are far more useful here. Look to c-sec vs. vag deliveries.
c3534l 9y ago
My mother went two years with a horrible stomach condition and couldn't get anyone to treat it until she wound up in the hospital and they were legally required to treat it. It was for exactly that reason: risky surgery looks bad, even if it's necessary. When there aren't a lot of doctors who can perform the surgery in question to begin with, say only two in your state, then you might just die.
lopmotr 9y ago
1/3 of heart surgeons violate medical ethics. That shows how flimsy and easily corrupted doctors are. That they don't really care about doing what's best for their patients' health as soon as an incentive makes that inconvenient. I hope some more data will reveal who has statistical anomalies in who they're turning away so the frauds can be identified.
flatfilefan 9y ago
Can there be positive self selection and specialization of the surgeons? Those who don’t believe they can probably operate successfully don’t do that at all and the others get more insensitive.
fallingfrog 9y ago
To be fair, you wouldn't want heart surgeons to shrug and take on operations they didn't feel ready for, either.
Dowwie 9y ago
I recommend listening to the new EconTalk episode about the tyranny of metrics
sunstone 9y ago
Fertility clinics are subject to the same pressures. They need to keep their success rates up so they avoid cases with poor prognoses.
oliwarner 9y ago
The only way this changes is removing the choice from the surgeon doing the cutting. Sounds a bit dramatic but if they think action is genuinely inappropriate, they should make the case to their colleagues, or a Multi Disciplinary Team, and let them have the final say. Together they should be able to enforce the Royal College and NICE guidelines for operating and the CQC —the main monitoring body— should be able to work out (both from data and on-the-ground inspectors) whether hospitals are doing as they should. I realise that's pretty UK specific but the ACS should be able to achieve something similar.

434 points by prostoalex 9y ago

Goodhart's Law again!

The subjective is always where there is value. I wish there were surgeons that would accept these challenges and colleagues would objectively assign a degree of difficulty score. And a site that aggregated such scores. I really don’t care how “friendly” a doctor is - did the get the diagnosis right? Did they improve longevity? Did they improve quality of life? Not easy to score but what everyone wants to know.

slap_shot 9y ago
Not sure why this is being downvoted. It’s a legitimate request. I worked for a novel medical device company and we had a lot of surgeons on staff. While not a product we’d ever build, there was always discussion amongst the doctors of how to better grade physicians and specifically surgeons. The concern is was that a lot, and I mean a LOT, of surgeons were bad at their job. They were always proposing different systems to evaluate surgeons, very close to what you described.
- ItsMe000001 9y ago
  > ... and colleagues would objectively assign a degree of difficulty score. He asks for a level of "objectivity" that you just can't get. What do you think medicine is? Like repairing toasters? "Why don't you simply assign some objective numbers" - it's so easy! No it is not, unless you only count a set of basic and routine procedures. Even things like a broken bone vary greatly - and then add the additional variance of the people having those broken bones. You can sort of get some objectivity when you do statistics of lots and lots of patients in many places over long periods - but the problem OP wants is to be able to make statements about individuals (doctors). Being able to say something about a population is very different from saying something about an individual within the population. "Statistics" about a single doctor are and will be full of randomness. Besides, if they did what I quoted from OPs statement above what would probably happen is that everybody declares everything as "very difficult" (or just below) expecting colleagues to do the same for them. In Germany private insurance has a multiplier on the amounts they can charge for something. Guess what: Everything is now declared as "very difficult" (with some short fuzzy explanation), multiplier 2.3. Almost nothing is charged at the "normal difficulty" rate. > Did they improve longevity? Did they improve quality of life? Given that each doctor is highly specialized, and that on the other hand your GP, the one who treats you long-term, can't actually do much when there is anything serious, those numbers are mostly out of the control of individual doctors. You would have to get such numbers system-wide, not per doctor. Medicine is supplied to patients throughout their life (incl. late life) by a network, not by an individual. There also is a huge impact of ones own life choices (e.g. obesity, smoking), and no, you can't just correct for them - not individually (for populations we can). All those numbers would look terrible for doctors who treat more of the worse cases. It would lead exactly to what the article is about. I'm not arguing against trying to measure surgeon outcomes - I'm against trying to use those measures for things they are not good for.
  - aantix 9y ago
    The feedback would have to be anonymous. It’s the only way to get honest scoring.
carbocation 9y ago
What you're describing is commonly referred to as "propensity weighting." It is a good first step, though most of us think it still doesn't do a sufficient job of accounting for the risk of difficult cases.
sky_rw 9y ago
Does the NHS provide enough free market choice that anybody could take advantage of a score even if they wanted to?
- dannyw 9y ago
  If you have private insurance, you can choose your own surgeon.
- ealexhudson 9y ago
  You can choose where to get treatment and who treats you; the main issue is whether you get the timeline you'd prefer. Generally, patients will choose a clinical center (to the extent that some hospitals have stopped offering specific treatments, because their stats were never good enough - e.g you need to do a certain number of children's heart ops per year to be good enough to get good stats). If you have the option to go private, either through your insurance or the NHS, then obviously there's another layer of choice. Most of the private surgeons are also NHS, anyway.
- SmellyGeekBoy 9y ago
  Yes, my father is currently undergoing treatment for cancer (involving multiple surgeries) and had issues with a surgeon. He was given the option to choose a different surgeon or even transfer to another hospital entirely.
- DanBC 9y ago
  It provides some. "Choose and Book" was introduced in 2004, and converted into e-referral in 2014, and it means you get a choice for most elective surgery. https://www.nhs.uk/NHSEngland/appointment-booking/Pages/about-the-referral-system.aspx https://www.nhs.uk/NHSEngland/appointment-booking/Pages/abou... https://www.england.nhs.uk/2014/05/choose-and-book/ https://www.england.nhs.uk/2014/05/choose-and-book/ Most people don't have enough information to make a sensible choice; and they don't understand the information they've got.

firstplacelast 9y ago

Check out the comments over on /r/medicine and you will likely be horrified. It is career over everything. There was even a thread this week about how you should NEVER correct someone above you (med students should never correct a resident and a resident should never correct an attending). That mentality is terrible for everyone and every thing, except the physicians' careers. I don't know if we need to select for better people when admitting them to medical school or if the culture is just completely fucked.

theatrus2 9y ago
Such practices were pervasive in other industries too, including air crews. Thankfully, this has been changing, and shown marked improvement in some cases.
Spooky23 9y ago
It’s a tough profession populated with type-a people, paid by insurance companies and beholden to malpractice insurers as well. As with anything in modern society, bean counters rule and usually drive awful decision making.
bradlys 9y ago
If you think about what it takes to get into med school then it seems pretty natural that you're going to get hyper competitive surgeons in the end. It starts before med school. You get assigned 10 unobtainable books that only the library has. Finals week comes around, all the books are missing even though they have to stay in the library. Turns out your classmates hid them inside the library so they could have a competitive advantage and get a higher grade on the curve. You already devoted probably half your life to becoming a surgeon, why would you screw it up and lose so much ground so easily?
- carbocation 9y ago
  What decade are you describing? In my medical school, all of the required material aside from our anatomy textbook was given to us as a PDF or a printout.

sebazzz 9y ago

Would it be possible to determine a "risk" factor and count that in the mortality ratings, or is this too simple?

MarkMc 9y ago
There is probably already some adjustment for risk - anyone can see that a smoker in his 60s will have higher mortality than a non-smoker in his 30s. The problem might be that the statistician doing the risk adjustment doesn't have 'skin in the game', so his/her assessment of risk is worse than the surgeon's. (Although if that were the case I would expect the statistician to be just as likely to overestimate the risk than underestimate it, yet I never hear about surgeons favouring the difficult cases because they think they can improve on the expected mortality rate in such cases).

carbocation 9y ago

The advent of public reporting of outcomes after intervention for myocardial infarction (heart attack) led to an increase in mortality in Massachusetts and New York.[1] 1 = https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4368858/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4368858/

fricat1ve 9y ago
A better way to make this claim would be to plot the mortality rates for the states over time and show that the rates in those places requiring reporting begin to diverge as the new policy is implemented.
- carbocation 9y ago
  They effectively did that: they compared public reporting states with other geographically adjacent states.
  - fricat1ve 9y ago
    There's nothing like Boston or NYC in the control states. If the effect is as big as they seem to think--a 21% increase in mortality--that should be visible on a plot of deaths over time.

tgraham 9y ago

66% of 115 specialists polled did not refuse a difficult operation. 33% thought about how it would impact their statistics, or put more generously, thought hard about the probability of the patient surviving presumably quite serious surgery. Yes mortality is a blunt statistic for a surgeon, but perhaps giving people pause for thought about the odds of success is no bad thing. Contra point: would you rather not know how many cases your surgeon has done that year, and the issues they have had?

maxerickson 9y ago
A large portion of the people undergoing heart surgeries still face a high risk of death if they don't have the surgery (with the surgery bringing the chance of a longer, more comfortable life). Here's why your hot take isn't the best take: “About 30 percent of them said they had turned patients down for surgery even when they knew full well that surgery was in their best interest.”
- TheRealPomax 9y ago
  Here's the converse of that, though: why do we think it's okay to expect a team of doctors and nurses to be okay with almost assuredly killing someone during surgery? They're not robots, "it's their job" is not an acceptable answer. Even if the surgery is in a patient's best interest, if the odds of killing them are all but guaranteed then it's most definitely not a matter of looking at the patient and rationalising it with "they will die otherwise anyway". It's not just about the patient. People who make this argument seem to forget that there's also an entire team of medical professionals that your rationalisation says should be okay with going into a surgery knowing they are almost guaranteed to kill this patient. They have the stats, the stats say "this person will die under the knife", many more lives are affected in this decision than just the patient. So expecting them to just do the surgery instead of going "No. This will kill the patient, I don't want that on me and my team" is very far from an okay attitude towards fellow human beings, and leads to terrible medical practices.
  - maxerickson 9y ago

billfruit 9y ago

On a related vein, do surgical procedures involve any type of quality control, where the work is independently audited? If so I wasn't aware. Quality control is such an important cogwheel of processes in many other industries, yet I don't hear much talk of it in relation to medical profession.

Dayshine 9y ago
Yes, at least in the UK, we have a series of national clinical audit programmes.
prepend 9y ago
There are process checklist reviews (http://qualitysafety.bmj.com/content/23/4/299 http://qualitysafety.bmj.com/content/23/4/299) and medical review panels (https://mrp.ky.gov/Pages/index.aspx https://mrp.ky.gov/Pages/index.aspx).
- deleted 9y ago
  [deleted]
fricat1ve 9y ago
In the US, if the surgery is in hospital: - Well-regarded hospitals will vet surgeons before granting privileges. - Average hospitals give out privileges fairly easily if there are no actions against a person's license. - There is a "collegial" review of big screw-ups that carry major reputational risk. - As with any fee-for-service firm, "rainmakers" are highly sought after and get away with more. - Privileges are difficult to take away once granted. (Have stronger legal protections than academic tenure in some states.) If the surgery is in an ambulatory surgery center or doctor's office: Basically anything goes. Ice-pick lobotomies, tonsillectomy mills, boob job factories in strip malls....all have happened in recent US history. About as well-regulated as traveling carnivals.
- billfruit 9y ago
  I was thinking something more akin to routine inspection program carried out by independent personnnel applicable to all procedures, not just the reviews that are necessitated due to screw ups.

drewg123 9y ago

It was not heart surgery, but after being diagnosed with lung cancer, my mother had one of her lungs removed by a surgeon who assured us he'd be able to get all the cancer. She had a miserable, painful recovery that I would not wish on anybody. Within a week or two of her recovering to the point where she could walk short distances, the cancer was found in her other lung and went on to kill her within a few more months. She may have gained several months of life from this operation, but it was a painful few months without dignity. It was 20 years ago, so my memory is fuzzy, but I don't remember ever being presented with anything but certainty from the surgeon that he could cure her. I think that patients and surgeons need to have a better idea of the realistic chance of success from a major surgery in order to make an informed decision. Perhaps if surgeons at least have to worry about their own numbers, they might hesitate before attempting cases like my mother's.

endymi0n 9y ago
How doctors die (2011): https://news.ycombinator.com/item?id=13122408 https://news.ycombinator.com/item?id=13122408
unpopular42 9y ago
That surgeon may have made a mistake, but you're suggesting to inject (more) unrelated incentives into this kind of decisions and expect them to get better? I'm sorry, but this sounds completely idiotic.
metaobject 9y ago
I guess I'm just shocked that the surgeon could offer that level of assurance about something like that.
- astura 9y ago
  Unbelievable! The five year survival rate for lung cancer is only 18%! Even when the cancer is localized the five year survival rate is only 55%![1] It's irresponsible for the doctor to not give a reasonable prognosis! [1]http://www.lung.org/lung-health-and-diseases/lung-disease-lookup/lung-cancer/resource-library/lung-cancer-fact-sheet.html http://www.lung.org/lung-health-and-diseases/lung-disease-lo...

samfisher83 9y ago

Sports athletes do this. For example players won't attempt end of the quarter half court shots since that will lower their percentages, or QBs taking sacks to not lower their QB ratings. What they do in sports is keep advanced stats. Instead of just giving a pure mortality rating have mortality rating, by surgery type.

deleted 9y ago
[deleted]
make3 9y ago
Then they just dodge the hard cases of each specific type of surgery. It's probably still an improvement though on just avoiding a surgery type altogether.
- killjoywashere 9y ago
  Then you track the deferral rates.
  - jessriedel 9y ago
    ...which punishes doctors for serving high-risk population.
  - TheCoelacanth 9y ago
    Then they start operating on people who don't really need it.
skookumchuck 9y ago
One could break out the quarter half court shots as a separate statistic.
- paulcole 9y ago
  At this point it’s very likely all teams do. The stats they’re using are nothing like what was printed on the back of basketball cards when we were kids.

trackofalljades 9y ago

Sounds a lot like police work in Japan (they love their stats).

cs702 9y ago

It's Goodhart's Law in action, this time with deadly consequences: "When a measure becomes a target, it ceases to be a good measure."[a] [a] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

batbomb 9y ago
Ironically applied to not-good hearts
- truculation 9y ago
  In another sense, 'Goodhart' seems like a splendid cosmic coincidence. Instead of obsessing over targets, try to perceive the whole system. And who can do that? Good-hearted individuals.
- mjfl 9y ago
  "These doctors are vile dogs. If I was a physician I would take all the hard patients, heal them all, and be a hero of the people." Yeah, if you were given one operation where the patient has a 100% chance of dying without it, but 80% of you killing them if you operate + 20% chance of them living after, maybe you'd do this once and get lucky. But try doing this twice, you have a a 96% chance of killing a patient, three times makes a 99.2% chance of killing at least a single patient. This probability quickly approaches 100%. If you kill a patient you get dragged into court, branded as a murderer by the opposing lawyer, and your career which you have spent your youth, your 20s and a half a million dollars training for is down the drain. Taking risky cases, if that's not your niche, is is guaranteed to have you out of a job and in debt from legal fees. To suggest that "bad hearts" are involved in this necessary calculation, is incredibly naive.
  - smnrchrds 9y ago
    I am pretty sure that the parent was referring to the fact that this article is about heart surgeries. Bad heart is used in the same manner as a bad knee or a bad tooth. It refers to the patient not the doctor. But aside from that, I completely disagree with your comment. The approach you are defending, even if legal, is completely unethical and immoral. It should be something we chastise, not condone.

deleted 9y ago

[deleted]

JumpCrisscross 9y ago

If you’re a patient, wouldn’t this still mean starting with the lowest-mortality surgeon? If they take your case, you’re least likely to die. If not, you have a better appraisal of your odds.

KMag 9y ago
That strategy may very well end up skipping (well, delaying selection) of a surgeon that routinely takes much harder cases than yours. I don't know enough about heart surgery to know if practice at more difficult procedures has any carryover to less difficult procedures, but it's conceivable . Your strategy might make sense, but it's not obviously optimal.
stordoff 9y ago
I'm not so sure. That might mean going with a surgeon where you are at the top end of their "risk profile", rather than a more skilled surgeon who has higher mortality rates due to routinely taking more difficult cases. I'm not sure you can generalise the mortality rate of all patients to your specific chance of death (a more skilled surgeon would likely be better for you, c.p.).

amelius 9y ago

Why don't we keep track of refusals as well?

B1FF_PSUVM 9y ago
Not a bad idea, per se, but then which is better - the doctor with 0% refusals and 20% deaths, or the one with 20% refusals and 10% deaths?
- wellboy 9y ago
  You compare it to the average death rate of a surgery.
- dredmorbius 9y ago
  Include the outcomes of the refused cases. Note that this also works to improve stats on docs who choose hard cases.

agumonkey 9y ago

it's been said a few times on /r/medicine, surgeons have their lawyer in mind when operating, they're skilled, but they first avoid anything that could damage their own life irreversably. Of course to an extent this goal aligns with the patient, but not always, and a bit sad on the ideal side.

kankroc 9y ago
Can someone really sue their surgeon in the US?
- agumonkey 9y ago
  I can't recall what countries these guys were.
- KMag 9y ago
  Absolutely. You can sue for pretty much anything. Winning is another story, but doctor's don't generally make sympathetic defendants. Juries know they have money and that they pay huge insurance fees to cover these cases. Of course, after losing a big case where the insurance company pays out millions of dollars, they likely become uninsurable and unemployable. But hey, the money is good until you hit the anti-lottery. My father was an anesthesiologist (who retired in good standing), and I remember him telling me a case he read about a Caesarean section. The surgeon was using an electrocautery pen to sear closed the ends of blood vessels. The surgeon set down the pen on the surgical cart, lifted the baby out, set the baby on the cart, and the baby's heel touched the pen. Now the baby has a small scar on the bottom of his/her heel for life. The parents sued the surgeon, the cart nurse overseeing the cart, and the anesthesiologist was close to the surgical cart. All 3 defendants settled out of court, since babies almost always win jury cases against rich doctors, regardless of merit. The litigant's lawyer would almost certainly bring up a below median survival rate as evidence of a pattern of gross incompetence. Also, of course, in the U.S., I believe only governments are immune to lawsuits (sovereign immunity) without their consent. The courts have consistently ruled that many rights, including the right to sue, cannot be legally waived by contract. In a similar way, if you sell yourself into slavery by entering into a contract that waives your right against unlawful detention, that contract cannot be legally enforced. My grandfather was an anesthesiologist. My dad was an anesthesiologist (and did well enough on the MCAT to go to med school after 3 years of undergrad... practiced medicine without an undergrad degree... med school is too competitive today to do such a thing). I was a good student: I graduated from MIT. However, my brother and I saw all of the BS and stress (anesthesiologists have a high suicide rate) and constantly rotating sleep/work shifts and both went into engineering.

trumped 9y ago

Some doctors probably refuse patients that left bad reviews at competing offices...

Mononokay 9y ago
"I don't appreciate the service they provide and they're terrible people !!!" "Okay, we won't serve you if you're just going to trash us." "EVIL HOW DARE YOU!"
- trumped 9y ago
  I meant if you left a bad review somewhere else... Of course someone should not go back where they don't like to go....
  - viraptor 9y ago
    That only works if you have more than one choice available.
    - trumped 9y ago
      The shitty US health/dental insurance system make it hard, but I still would be surprised if most people don't have a choice of more then one doctor...

Numberwang 9y ago

It's sad to hear this when it's totally unnecessary. Hear disease has been a solved problem for decades. Just compare the rates of disease for vegans with the general population.

brookside 9y ago
What are the rates?
- Numberwang 9y ago
  http://www.mercyforanimals.org/american-heart-association-vegan-diet-decreases http://www.mercyforanimals.org/american-heart-association-ve...
  - brookside 9y ago
    > Researchers studied five different diets to see which lowered the risk of heart failure: convenience (fast food and pasta), plant-based, sweets, Southern (sugary beverages and fried foods), and alcohol/salads. What kind of study design is this? It seems like they compared veganism with 4 garbage diets. Seems like all we can extract from this is plant-based is better than eating trash.
danans 9y ago
> Just compare the rates of disease for vegans with the general population. "Disease" is such a broad category that it's sort of meaningless here. Maybe veganism is healthier than an animal diet, but the degree of its impact can only be stated only after removing the many other environmental factors that might be contributing to lower disease rates among vegans, including income and location.
- Numberwang 9y ago
  Yes, lot's of things are always involved. Including air quality etc as I think I saw on HN just the other day. I find it interesting though that the right type of vegan diet will basically make you hypertension and heart attack proof and that it may also reverse heard disease (the only diet to do so). Especially when you look at mortality and costs of healthcare this really need to be more generally recognized.

NTDF9 9y ago

Reminds me of why some devs will not touch critical pieces of code because the next time a bug comes in that module, git log shows they were last editors and will be asked to take a look (even if the bug is unrelated to them)

narrator 9y ago

They need to add a difficulty correlated metric to the statistics. Maybe BMI and age? I am sure IVF clinics wouldn't offer the procedure to anyone over 35 if the rather detailed statistics they are required to report didn't also include the patient's age.

Lxr 9y ago
I imagine it’s extremely difficult to come up with a metric that captures everything about the case. BMI, age, tumor type, tumor shape, history of kidney disease, diabetes, where do you stop?
- MarkMc 9y ago
  You don't need a metric which captures everything about the case - you just need a statistical model whose risk assessment is at least as accurate as the surgeon's assessment. This is not as difficult as it sounds - in Chapter 21 of Thinking Fast and Slow, Kahneman makes a strong case that simple algorithms are very often better than expert clinical judgement.
zzbzq 9y ago
Rather than try to figure out some classification criteria, use a predictions market. Have surgeons predict their own chance of success. Preferably, gather multiple second++ opinions for each case. The difficulty of the surgery is based on some combination of the different surgeon's predictions. A surgeon's stats would then be based on his successful and unsuccessful surgeries, weighted against the difficulty. Using this method, not only can you still rank the surgeons based on the successes of their surgeries, but also on the accuracy of their predictions, which will be in larger sample size and also perhaps a better indicator of their mastery of medical knowledge.

Nuzzerino 9y ago

Can we get a (2016) on the thread title?

SilasX 9y ago

We bundle up too many conflicting jobs into the role of the doctor. Rating the difficulty and status of an incoming patient should be completely separate from the job of treating them, and all judgments about doctor effectiveness should be judged relative to those assessments. (Credit: alerted to this misincentive by Yudkowsky’s recent book Inadequate Equilibria.)

adolph 9y ago

This topic was discussed in the last EconTalk [1] podcast featuring the author of a recent book “The Tyranny of Metrics” [2]. I think one of the author’s interest insights is the distinction between metrics for diagnostics and those for incentives. 1. http://www.econtalk.org/archives/2018/04/_what_is_the_ap.html http://www.econtalk.org/archives/2018/04/_what_is_the_ap.htm... 2. https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller-ebook/dp/B076ZWW2MN https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller-ebook/...

TwoBit 9y ago
It's what I hate about the legal system. Prosecutors don't care about justice, they care about a record of putting thevmost people away.

skookumchuck 9y ago

One possibility is to offer the surgeons more money for difficult surgeries, and more money for a successful outcome, to balance out the reputational risk to the surgeon.

dfox 9y ago
How do you estimate the difficulty in a way that does not involve wrong incentives on the part of the surgeon or whoever is doing the estimation when money is involved?

2skep 9y ago

What would you prefer? A surgeon who expects a poor outcome of your surgery: a)decides to not operate on you? b)decides to operate on you nonetheless? I am in camp a

calcifer 9y ago
I'm definitely in camp B. 90% chance of dying on the table beats a 100% chance of dying without the surgery, every single time.
- jack9 9y ago
  Quality of life matters in most of these marginal cases. If they are going to give you the Konno procedure (scrape extra tissue out of a thickened ventricle), there's tons of things that can go wrong OTHER than death. Death is assured, either way, for everyone.
stordoff 9y ago
It depends what the outcome is without the surgery. If I'm dead without it, I want them to operate regardless. If there's only a chance of death, or the surgery is aiming to improve quality of life (rather than prevent [imminent] death), it's more of a judgement call.

emodendroket 9y ago

You would think that they could adjust these figures by patient age and condition.

crusso 9y ago

Ultimately, we need to compare apples to apples by building rating systems that account for the difficulty of treating patients with various, ailments at various ages, with various complications. Think of it in terms of competitive diving, gymnastics, or snowboarding: Your scores depend upon the difficulty of the moves you attempted combined with your performance of those specific moves. The more difficult the move, the more you're compensated for even attempting it when it comes to scoring. The trick then is to prevent physicians from gaming the system by exaggerating the difficulty of the patients they're dealing with. You'd need to use as many objective metrics as possible and possibly some system of having different physicians assess ratings than the ones performing the procedures - maybe in some kind of double-blind fashion to prevent any kind of coordination strategy.

calcifer 9y ago
> some system of having different physicians assess ratings than the ones performing the procedures - maybe in some kind of double-blind fashion to prevent any kind of coordination strategy. That would simply result in an unwritten rule where everyone gives everyone else good ratings.
fricat1ve 9y ago
The "objective metrics" part is key. Medical records are generally written in a way that weaves a standard narrative of the doctor as either a hero/savior or a valiant defender of a lost cause. Doctors only get investigated when derogatory facts accumulate to an extent that this "standard narrative" collapses. It's not really an objective or scientific standard.
vannevar 9y ago
The article itself alludes to a 'EuroSCORE' that purports to rate the risk of mortality beforehand. It seems like the mortality ratings should be risk-adjusted. A double-blind system as you suggest could work like this: First, the rating system would be opt-in on the part of the participating surgeons. It seems likely that patients would prefer rated surgeons, so there would be an incentive to participate but it would not be mandatory. Those who belonged to the rating body would be required to do anonymous risk assessment on some number of cases per month. They wouldn't know who the patients are or who is treating them, they would only get anonymized records. Their risk assessments would be calibrated in the short run against other assessments, and in the long run against outcomes. These calibrated risk assessments would be used to adjust the score of the treating surgeon on the outcome, so that they would be penalized less for mortality in patients with a very high assessed risk.

martyvis 9y ago

In Australia, we have a neurosurgeon Charlie Teo that is well known for operating on "inoperable" brain tumours. While he has a lot of respect in the general community, to many of his surgical peers he is seen as a reckless maverick. I would think his stats on average are bad, but because he will tackle outliers he has won the support of the people that matter - his patients.

wastra 9y ago

The incorrect assumption your readers are making is that heart surgery is a good thing in all cases. America is the leader in futile medical procedures: it may well be a good thing to have fewer risky procedures on patients who were more likely to be harmed. "primum no nocere" may be advanced!

tyingq 9y ago

Metrics do drive behavior. Like, in the IT space, percentage of successful change requests. Encouraged you to power though even if things look bad. Because aborting is a 100% chance of a ding to the metric, while taking the risk is something less. The stakes are lower, of course, but the idea is similar.

paulcole 9y ago

This exact topic was discussed in the 1968 novel Case of Need by Jeffrey Hudson. It was his first book, written while a medical student at Harvard. The novel raised a stir as it was critical of Harvard faculty and in some ways the practice of medicine in general. The book eventually won an Edgar Award. Some may know Jeffrey Hudson better by his given name, Michael Crichton. https://en.m.wikipedia.org/wiki/A_Case_of_Need https://en.m.wikipedia.org/wiki/A_Case_of_Need

tomohawk 9y ago

Relatedly, I've met many people who had a surgery where they used to live and, after moving, other surgeons won't touch them. They have to travel back to where they used to live to get care. Seems really unethical.

probo 9y ago

You cannot compare crude outcomes. You have to adjust for so called "case-mix". For mortality after cardiac surgery, the risk can be calculated using the EuroSCORE II. http://www.euroscore.org/calc.html http://www.euroscore.org/calc.html And even then, often the numbers for individual surgeons are too small to draw conclusions.

qrbLPHiKpiux 9y ago

A colleague of mine was interviewed by a new hospital heart program. He was asked what his mortality rate was. He replied, what do you want it to be?

SteveGerencser 9y ago

I firmly believe that this is why my mother-in-law died. She needed surgery to fix a hernia that basically left her unable to eat at all. Is she did not have the surgery she was going to die. The surgeon stopped the surgery because the 'risk' was too high. 48 hours later she was dead.

hycaria 9y ago
You don't die in 48h from not eating. So many elements missing there, this is such a terrible depiction. I hope anyone passing by will have a bit of critical thinking.

imaadrashied 9y ago

This isn't surprising because humans will do what they're incentivized to do (speaking mainly in a professional setting here). Might be effective to add a measure to the transparency that shows something to the effect of, "likelihood to perform life-saving and risky procedures." Therefore a high win-loss with a lower risk threshold would be weighted lower than a so-so win-loss with a higher risk threshold.

ianformanek 9y ago

There might be several decent ways to improve (at least as a next step) to avoid this particular way to game the KPIs: - use an assessed upfront difficulty (if available in measurable form) to adjust for the outcome rating - add additional KPI for % of refused operations (perhaps again adjusted for difficulty)

justonepost 9y ago

Alternatively, you have a good idea of what your mortality rate will be if a surgeon picks you for a case.

txsh 9y ago

Should “NHS” be added to the title? This seems to only apply to British heart surgeons.

mitko 9y ago

Shameless plug - I wrote an essay recently on how any metric becomes a vanity metric http://dimitarsimeonov.com/2018/03/22/the-vanity-metric-paradox http://dimitarsimeonov.com/2018/03/22/the-vanity-metric-para... Some excerpts: - having clear, well defined metrics is the single largest driver of progress within a company - Any metric sufficiently optimized becomes a vanity metric - Building products is not science. What makes a good product is usually dependent on so many factors that are subject to change and evolve. - Over time, a product no longer stands to die if these metrics degrade. - According to EdwardTufte - people and institutions cannot keep their own score Hope someone finds it useful!

randyrand 9y ago

Easily solvable. Have a person whos jobs it is to rank how likely a person is to live. He gets paid based on how accurate he is. Doctors would then be ranked as an offset from this previous mortality prediction.

Consultant32452 9y ago
This incentivizes the doctor to maximize the appearance of risk in the medical record.
- Too 9y ago
  The one doing the estimate should obviously not be the same doctor doing the surgery.
  - Consultant32452 9y ago
    The one doing the estimate will do so based off the records generated by the doctor doing the surgery.
    - dredmorbius 9y ago
      Not for referrals.
      - Consultant32452 9y ago
        The only way to get reasonably accurate results with this is to have two surgeons do completely separate diagnostics in a double-blind where neither they nor the patient knows which one is going to be performing the surgery. That seems unworkable at scale.
        dredmorbius 9y ago
        Surgeons don't do diagnostics, as a general rule. Internists, paths, triage, GP, trauma do.

dosycorp 9y ago

That means there's an opportunity / gap in the market. Demand for risky operations by motivated buyers, should lead to some doctors taking on the role / label of risk-takers / explorers. Since they can get the customers other, risk averse doctors are turning away, and pull in rare-treatment seekers on their own.

fedups 9y ago

Econtalk had a good interview this month on this general phenomenon discussing the book "The Tyranny of Metrics" http://www.econtalk.org/archives/2018/04/jerry_muller_on.html http://www.econtalk.org/archives/2018/04/jerry_muller_on.htm...

deleted 9y ago

[deleted]

refurb 9y ago

This is the classic problem where people use averages as a metric, but ignore the fact that variations are not distributed evenly. I remember looking at the economics of a clinic for our product. Someone said "let's just give them a 5% discount as on average they will break-even. That works if you look at on average how much they make, but ignore the fact that every clinic has a different mix of insurance companies and *their own economics are all different."

acjohnson55 9y ago

We chose to have our daughter delivered at a hospital that is renowned for its neonatal ICU. It's obviously a great place to give birth, but if you look on paper, their c-section and other intervention rates are above average. Which makes sense when you understand that they are equipped to handle the highest risk pregnancies. A similar phenomenon occurs in education. I taught high school in Baltimore, where the vast majority of my students were high risk. Baltimore has a few renowned high schools, but by and large, they serve the easiest students.

stevenwoo 9y ago

There was a counterintuitive study of obstetricians that I cannot find right now that showed that in one state over many (10?) years the ones who did the most C-sections were the best doctors - because when researchers looked at the underlying data the realized these doctors treated the patients who had the worst predicted outcomes and beat the national odds - I think they theorized that these doctors had the best ability to choose the optimal method of birth and were not afraid to choose c-section when it was necessary.

dredmorbius 9y ago
There's a very strong relationship between experience (number of procs performed) and outcomes. High-volume is almost always beetter. This may of course favour unnecessary ops...)
- stevenwoo 9y ago
  Yes, I think this was particularly against the grain because mothers are commonly told to look for a doctor/practice who does not resort to a C-section right away, and judge this by the number of C-sections a doctor/practice performs.
  - dredmorbius 9y ago
    Hospital rather than doc rates are far more useful here. Look to c-sec vs. vag deliveries.

c3534l 9y ago

My mother went two years with a horrible stomach condition and couldn't get anyone to treat it until she wound up in the hospital and they were legally required to treat it. It was for exactly that reason: risky surgery looks bad, even if it's necessary. When there aren't a lot of doctors who can perform the surgery in question to begin with, say only two in your state, then you might just die.

lopmotr 9y ago

1/3 of heart surgeons violate medical ethics. That shows how flimsy and easily corrupted doctors are. That they don't really care about doing what's best for their patients' health as soon as an incentive makes that inconvenient. I hope some more data will reveal who has statistical anomalies in who they're turning away so the frauds can be identified.

flatfilefan 9y ago

Can there be positive self selection and specialization of the surgeons? Those who don’t believe they can probably operate successfully don’t do that at all and the others get more insensitive.

fallingfrog 9y ago

To be fair, you wouldn't want heart surgeons to shrug and take on operations they didn't feel ready for, either.

Dowwie 9y ago

I recommend listening to the new EconTalk episode about the tyranny of metrics

sunstone 9y ago

Fertility clinics are subject to the same pressures. They need to keep their success rates up so they avoid cases with poor prognoses.

oliwarner 9y ago

The only way this changes is removing the choice from the surgeon doing the cutting. Sounds a bit dramatic but if they think action is genuinely inappropriate, they should make the case to their colleagues, or a Multi Disciplinary Team, and let them have the final say. Together they should be able to enforce the Royal College and NICE guidelines for operating and the CQC —the main monitoring body— should be able to work out (both from data and on-the-ground inspectors) whether hospitals are doing as they should. I realise that's pretty UK specific but the ACS should be able to achieve something similar.