10 ms·

I'm astounded at the tenor of comments on this article, which I would expect to find at any site other than this one. Yes, measuring surgical outcome data and

I'm astounded at the tenor of comments on this article, which I would expect to find at any site other than this one. Yes, measuring surgical outcome data and surgeon performance could lead to a variety of complications, perverse incentives, regulatory capture, etc. etc. All true. But you know what's worse than that? Having no fucking information about anything. I'm in the process of trying to find a surgeon to help with an orthopedic procedure, and it is so incredibly frustrating that I have a thousand times as much information on which refrigerator I should buy, or which phone, than about something that will affect my life more intimately than any consumer product ever could. And yes, of course I get that rating a commodity with a fixed function that is used by zillions of people is way more straightforward. But look, all of these apocalyptic scenarios about brain surgeons who only take trivial cases so they can have a better score on the leaderboard? I'm just not worried about it. At all. Because people are generally proud and want to do better. Because of the social and professional stigma that would come from such behavior. Because if you collect a rich dataset, you can account for most kinds of gaming, just as you can do for teacher outcomes, or every other damn thing. But probably most of all because the current state of affairs is so abjectly wretched, and literally any effort at measurement and accountability would be better under every reasonable scenario.

moonka 12y ago
This seems to be the case for everything to do with Healthcare. Even choosing a healthcare plan is confusing and terrifying, and then figuring out if what you need is covered or not is hard to do.
arjie 12y ago
Certainly you may not be worried about this, but when there are human beings involved one has to be more careful. With any physical process, you can change something by just applying force. Changing the status quo in situations like these is harder. The harder you push, the more they'll form a rock wall against your demands. Nobody wants to protect the poor surgeon. But everyone knows the tragic fact: each one of them was a poor surgeon at some procedure at some point. Experience made them better. The article stresses the importance of the way these things are done. In order to have surgeons participate you need to earn their trust. Show them that you're not out to 'expose the failures' so to speak. Besides, the idea that surgeons will only take easy cases is not a hypothetical. The article lists instances where gaming occurred. In fact, it stresses the importance of confidentiality in gaining surgeons' trust in ensuring this doesn't happen. Like in everything else, we should treat how to change systems as a science and the evidence looks like it favours Amplio's approach to the problem. Maybe in some years from now things will be different and more information can be public. But for that to succeed we need to get there carefully.
- hueving 12y ago
  Taking easy cases could easily be accounted for by tracking the cases the surgeon chose not to take as well.
patcheudor 12y ago
The thing is, many of us have been around the block on more than one occasion with poorly thought out metrics which lead to unintended and down right bad consequences. A few years ago I was rallying against a system used to rate and rank relative risk of software systems based on a score. Many times I was told: "at least it's better than nothing." Ultimately it was discovered that there was nearly an inverse relationship to the scoring. Those bits of software which ranked as the most secure ended up being the least, with the least secure being the most. This happened because developers unaware of security concepts assumed security was happening elsewhere rather than taking the initiative to practice secure coding. Interestingly enough this entirely invalidated the argument that doing something was better than nothing as the system drove misplaced focus and overall reduced the security of the overall environment as the number of bad bits of software out numbered the good, thus leading to poor hiring practices. Management saw the score and figured everything was getting better and the environment needed fewer security resources rather than the reality which was that many more were needed. Complex systems are challenging & one study isn't going to provide the miracle solution. Surgery is not a refrigerator and to think every problem can immediately be solved by a trivial metric can be a dangerous game, especially when it comes to lives. When it comes to health care I tend to look at the macro, rather than the micro. Which hospitals have higher ratings, lower insurance claims and rates, etc., however, I understand that information can be gamed as well and take it with a grain of salt. At the end of the day I hedge my bets in this space by eating well, working out daily, and otherwise trying to avoid the medical industry at all costs.
- nostromo 12y ago
  So we measure nothing? As pointed out in your comment, the fix is the consumer of the information understanding it and using it correctly, not avoiding the data entirely. This reminds me of the debate about improving schools in the U.S. We hear objections from teachers unions how it's literally impossible to measure the efficacy of schools and teachers. What other industries can afford to not rate their employees' effectiveness, outside of the broken markets of primary education and healthcare?
  - fixedd 12y ago
hermanhermitage 12y ago
Incase you find this useful: Just lying in hospital bed here about 36 hours after 2nd revision on a THR first done in 1987. In the absence of clear public metrics I used the following approach to choose my most recent surgeon: 1. Age/experience. I was looking for someone about 2/3 of their way thru career. Assuming experience and survival bias were positive indicators. 2. I got a series of second opinions asking surgeons the direct question of whom they would get to operate on themselves if they needed the procedure done. 3. I consulted a series of family doctors on their recommendations. 4. I went with someone with a strong track record designing prostheses but also a large history of performing the particular procedure I was having. There are always trade offs. My surgeon works long shifts and so I got them 12 hours into a shift on a friday, which had me a little nervous (fatigue wise). So my main advice is don't be shy in getting a lot of opinions and advice. I will probably need another 2-3 revisions in my life time on this hip alone. So I thought it worth my while taking a thorough approach. Still at the end of the day bad luck can always happen. Expect the best but be mentally prepared for the worst.
- tomcam 12y ago
  I believe #2 is key and have thought for years this could be the basis of a super-effective site. But not a lucrative one: a key problem is that any kind of monetization that would benefit the doctors doing the rating could pervert the results.
  - imaginenore 12y ago
    It could easily turn into "i scratch your back, you scratch mine" type of reviews. Even voting rings of doctors.
    - hermanhermitage 12y ago
      Definitely detected that in my travels - diabolical surgeon golfers ring :). I learnt to ask them what handicap they play off.
- shanusmagnus 12y ago
the_cat_kittles 12y ago
i think the kind of doctors that would "game the system" are probably already shitty doctors, so i think the risk of surgeons only taking trivial cases is even less of a big deal.
teddyh 12y ago
> But you know what's worse than that? Having no fucking information about anything. Sometimes, having no information, and choosing truly randomly, can be better than having bad information and choosing according to your own biased and faulty intuition and preconceptions.
- schuman 12y ago
  In that case, how about you make decisions without information, let others get and use information, and we'll let Darwin sort this shit out.
jacalata 12y ago
>Because if you collect a rich dataset, you can account for most kinds of gaming, just as you can do for teacher outcomes, or every other damn thing. Has this actually been solved for teacher outcomes or are you just saying it could be?
noonespecial 12y ago
It may conceivably be a feature and not a bug at least as American healthcare is implemented today. Imagine if was known beyond doubt who the single best surgeon in the country was (and people were more aware of just how big a difference in outcome surgeon skill makes). How many people would demand and feel entitled to that surgeon? Who would shrug and say "oh well, I'm on Medicare so 146th best will be fine"? There's a myth floating around today that everyone is entitled to and can have "the very best" medical care. (Or this is the way it would be except demo/publicans.) Objective scores would severely damage this myth.
- refurb 12y ago
  You make a great point. At the same time the US is already kind of there. In Canada most folks just go to their local hospital. Only if that hospital can't provide care do you go somewhere else. In the US, i've noticed that folks often pursue the best care they can, especially if it's a serious condition. Of course that's if you're insurance covers it.
k-mcgrady 12y ago
Is 'choosing a surgeon' an American thing or a private health care thing? Personally I've I'm ill enough to be in hospital I get admitted, they assign me a consultant and if necessary he assigns me a surgeon. No choosing involved.

moonka 12y ago

This seems to be the case for everything to do with Healthcare. Even choosing a healthcare plan is confusing and terrifying, and then figuring out if what you need is covered or not is hard to do.

arjie 12y ago

Certainly you may not be worried about this, but when there are human beings involved one has to be more careful. With any physical process, you can change something by just applying force. Changing the status quo in situations like these is harder. The harder you push, the more they'll form a rock wall against your demands. Nobody wants to protect the poor surgeon. But everyone knows the tragic fact: each one of them was a poor surgeon at some procedure at some point. Experience made them better. The article stresses the importance of the way these things are done. In order to have surgeons participate you need to earn their trust. Show them that you're not out to 'expose the failures' so to speak. Besides, the idea that surgeons will only take easy cases is not a hypothetical. The article lists instances where gaming occurred. In fact, it stresses the importance of confidentiality in gaining surgeons' trust in ensuring this doesn't happen. Like in everything else, we should treat how to change systems as a science and the evidence looks like it favours Amplio's approach to the problem. Maybe in some years from now things will be different and more information can be public. But for that to succeed we need to get there carefully.

hueving 12y ago
Taking easy cases could easily be accounted for by tracking the cases the surgeon chose not to take as well.

patcheudor 12y ago

The thing is, many of us have been around the block on more than one occasion with poorly thought out metrics which lead to unintended and down right bad consequences. A few years ago I was rallying against a system used to rate and rank relative risk of software systems based on a score. Many times I was told: "at least it's better than nothing." Ultimately it was discovered that there was nearly an inverse relationship to the scoring. Those bits of software which ranked as the most secure ended up being the least, with the least secure being the most. This happened because developers unaware of security concepts assumed security was happening elsewhere rather than taking the initiative to practice secure coding. Interestingly enough this entirely invalidated the argument that doing something was better than nothing as the system drove misplaced focus and overall reduced the security of the overall environment as the number of bad bits of software out numbered the good, thus leading to poor hiring practices. Management saw the score and figured everything was getting better and the environment needed fewer security resources rather than the reality which was that many more were needed. Complex systems are challenging & one study isn't going to provide the miracle solution. Surgery is not a refrigerator and to think every problem can immediately be solved by a trivial metric can be a dangerous game, especially when it comes to lives. When it comes to health care I tend to look at the macro, rather than the micro. Which hospitals have higher ratings, lower insurance claims and rates, etc., however, I understand that information can be gamed as well and take it with a grain of salt. At the end of the day I hedge my bets in this space by eating well, working out daily, and otherwise trying to avoid the medical industry at all costs.

nostromo 12y ago
So we measure nothing? As pointed out in your comment, the fix is the consumer of the information understanding it and using it correctly, not avoiding the data entirely. This reminds me of the debate about improving schools in the U.S. We hear objections from teachers unions how it's literally impossible to measure the efficacy of schools and teachers. What other industries can afford to not rate their employees' effectiveness, outside of the broken markets of primary education and healthcare?
- fixedd 12y ago

hermanhermitage 12y ago

Incase you find this useful: Just lying in hospital bed here about 36 hours after 2nd revision on a THR first done in 1987. In the absence of clear public metrics I used the following approach to choose my most recent surgeon: 1. Age/experience. I was looking for someone about 2/3 of their way thru career. Assuming experience and survival bias were positive indicators. 2. I got a series of second opinions asking surgeons the direct question of whom they would get to operate on themselves if they needed the procedure done. 3. I consulted a series of family doctors on their recommendations. 4. I went with someone with a strong track record designing prostheses but also a large history of performing the particular procedure I was having. There are always trade offs. My surgeon works long shifts and so I got them 12 hours into a shift on a friday, which had me a little nervous (fatigue wise). So my main advice is don't be shy in getting a lot of opinions and advice. I will probably need another 2-3 revisions in my life time on this hip alone. So I thought it worth my while taking a thorough approach. Still at the end of the day bad luck can always happen. Expect the best but be mentally prepared for the worst.

tomcam 12y ago
I believe #2 is key and have thought for years this could be the basis of a super-effective site. But not a lucrative one: a key problem is that any kind of monetization that would benefit the doctors doing the rating could pervert the results.
- imaginenore 12y ago
  It could easily turn into "i scratch your back, you scratch mine" type of reviews. Even voting rings of doctors.
  - hermanhermitage 12y ago
    Definitely detected that in my travels - diabolical surgeon golfers ring :). I learnt to ask them what handicap they play off.
shanusmagnus 12y ago

the_cat_kittles 12y ago

i think the kind of doctors that would "game the system" are probably already shitty doctors, so i think the risk of surgeons only taking trivial cases is even less of a big deal.

teddyh 12y ago

> But you know what's worse than that? Having no fucking information about anything. Sometimes, having no information, and choosing truly randomly, can be better than having bad information and choosing according to your own biased and faulty intuition and preconceptions.

schuman 12y ago
In that case, how about you make decisions without information, let others get and use information, and we'll let Darwin sort this shit out.

jacalata 12y ago

>Because if you collect a rich dataset, you can account for most kinds of gaming, just as you can do for teacher outcomes, or every other damn thing. Has this actually been solved for teacher outcomes or are you just saying it could be?

noonespecial 12y ago

It may conceivably be a feature and not a bug at least as American healthcare is implemented today. Imagine if was known beyond doubt who the single best surgeon in the country was (and people were more aware of just how big a difference in outcome surgeon skill makes). How many people would demand and feel entitled to that surgeon? Who would shrug and say "oh well, I'm on Medicare so 146th best will be fine"? There's a myth floating around today that everyone is entitled to and can have "the very best" medical care. (Or this is the way it would be except demo/publicans.) Objective scores would severely damage this myth.

refurb 12y ago
You make a great point. At the same time the US is already kind of there. In Canada most folks just go to their local hospital. Only if that hospital can't provide care do you go somewhere else. In the US, i've noticed that folks often pursue the best care they can, especially if it's a serious condition. Of course that's if you're insurance covers it.

k-mcgrady 12y ago

Is 'choosing a surgeon' an American thing or a private health care thing? Personally I've I'm ill enough to be in hospital I get admitted, they assign me a consultant and if necessary he assigns me a surgeon. No choosing involved.