4 ms·

Sorry yes, I accept it is more complex than I described. There is the possibility that (a) the hospital admitting a patient can assess patient risk more accurat

by MarkMc 10y ago

Sorry yes, I accept it is more complex than I described. There is the possibility that (a) the hospital admitting a patient can assess patient risk more accurately than a university professor who specialises in risk factors; (b) the size of the hospital advantage is significant when compared to the real differences in risk between hospitals; and (c) the hospital takes advantage of its better analysis by turning away patients where the professor risk rating of the patient is higher than the hospital risk rating. This is subtly different to your analysis in two ways: 1. The trouble with using survival or outcome ratings is that they effectively punish providers who take on the hardest, riskiest cases. Not necessarily - if the professor underestimates the risk in easy cases and overestimates the risk in hard cases it would give the hospital an incentive to take on the hard cases and turn away the easy cases. 2. The reality is that's it's impossible to accurately quantify all the different factors which potentially impact survival. In some cases we have remarkably detailed understanding of a broad range of risk factors - take a look at table 3 from page 23 in this [1] report on stem cell transplants. There are detailed odds ratios for patient age, ethnicity, karnofsky score, CMV status, disease type and stage, interval between diagnosis and transplant, year of transplant, donor match type, donor/recipient sex parity and donor age. More importantly, the professor doesn't need to be particularly accurate in assessing patient risk - she just needs to be as accurate as the hospital. (Or close enough that the difference would not give the hospital a significant advantage) Can the professor get close enough to hospital accuracy when estimating patient risk? Two relevant points spring to mind: (a) In the book Thinking Fast and Slow there is a description of how expert analysis can often be matched by simple rules of thumb. (b) The book Superforcasting describes how some lay people are able to make better predictions than the CIA about future geopolitical events, despite having no access to classified information. My gut feel is that the hospital would not have a significant edge over the professor. But I'm willing to be proved wrong - let's conduct an experiment where a doctor estimates the 3-year chance of survival for each patient at the time the patient is admitted to hospital, and a professor also makes the same estimate based only on the patient's database record. The winner gets half a million dollars. Wouldn't this experiment give us valuable insight in determining whether hospitals can game an End Results system? [1] http://www.seattlecca.org/sites/default/files/content_page/2016-07/9f/inline-documents/CIBMTR-Final-2015-Transplant-Center-Specific-Survival-Report.pdf http://www.seattlecca.org/sites/default/files/content_page/2...

nradov 10y ago
No rational hospital administrator would ever agree to be judged on 3-year survival rates. The sample sizes are too small and there are too many uncontrollable confounding factors impacting survival for this to ever give statistically significant results. Did the patient die because his heart bypass surgeon was incompetent, or because he went right back to eating Big Macs as soon as he was discharged from the hospital?
- MarkMc 10y ago
  Well there are about 370 hospitals who agree to have patient outcomes on cardiac surgery published [1]. And there are 173 hospitals who agree to have patient outcomes on stem cell transplants published [2]. And if we keep the identity of the hospital confidential, the number of hospitals willing to provide data on patient outcome grows substantially. The study referenced by the New York Times article [3] had 22 million hospital admissions, and the Cystic Fibrosis Foundation has gathered detailed data from all US Cystic Fibrosis treatment centers for 50 years [4]. Although it would be better for all hospitals to be completely transparent, there are still significant benefits to judging hospitals in anonymous fashion, as described in [4] (which is a fascinating article in its own right). As for statistically significant results, it's true that we will be unable to judge the effectiveness of smaller hospitals (unless they are extremely good or extremely bad). But for medium-sized and large hospitals this is not a problem - in the stem cell study [3] there were 43 out of 173 hospitals whose 2015 results were judged to be outside the 95% confidence interval (26 were better than predicted to a statistically significant degree, while 17 were worse). Of those 43 outliers, 26 were also outliers in 2014 which suggests this is not just a fluke. Regarding your question: "Did the patient die because his heart bypass surgeon was incompetent, or because he went right back to eating Big Macs as soon as he was discharged from the hospital?": Keep in mind that we can control for body-mass index, so the question is whether people with similar BMI will be more likely to eat Big Macs after treatment at Hospital A, rather than Hospital B? Yes it's possible, but it seems to me more likely to be an effect of the hospital. Also note that it may not be down to the quality of the heart bypass surgeon - it may be simply that Hospital A is more effective at encouraging healthier eating after patients are discharged. [1] http://www.sts.org/quality-research-patient-safety/sts-public-reporting-online http://www.sts.org/quality-research-patient-safety/sts-publi... [2] https://www.seattlecca.org/sites/default/files/content_page/2016-07/9f/inline-documents/CIBMTR-Final-2015-Transplant-Center-Specific-Survival-Report.pdf https://www.seattlecca.org/sites/default/files/content_page/... [3] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0166762 http://journals.plos.org/plosone/article?id=10.1371/journal.... [4] http://www.newyorker.com/magazine/2004/12/06/the-bell-curve http://www.newyorker.com/magazine/2004/12/06/the-bell-curve

by MarkMc 10y ago

nradov 10y ago

No rational hospital administrator would ever agree to be judged on 3-year survival rates. The sample sizes are too small and there are too many uncontrollable confounding factors impacting survival for this to ever give statistically significant results. Did the patient die because his heart bypass surgeon was incompetent, or because he went right back to eating Big Macs as soon as he was discharged from the hospital?

MarkMc 10y ago
Well there are about 370 hospitals who agree to have patient outcomes on cardiac surgery published [1]. And there are 173 hospitals who agree to have patient outcomes on stem cell transplants published [2]. And if we keep the identity of the hospital confidential, the number of hospitals willing to provide data on patient outcome grows substantially. The study referenced by the New York Times article [3] had 22 million hospital admissions, and the Cystic Fibrosis Foundation has gathered detailed data from all US Cystic Fibrosis treatment centers for 50 years [4]. Although it would be better for all hospitals to be completely transparent, there are still significant benefits to judging hospitals in anonymous fashion, as described in [4] (which is a fascinating article in its own right). As for statistically significant results, it's true that we will be unable to judge the effectiveness of smaller hospitals (unless they are extremely good or extremely bad). But for medium-sized and large hospitals this is not a problem - in the stem cell study [3] there were 43 out of 173 hospitals whose 2015 results were judged to be outside the 95% confidence interval (26 were better than predicted to a statistically significant degree, while 17 were worse). Of those 43 outliers, 26 were also outliers in 2014 which suggests this is not just a fluke. Regarding your question: "Did the patient die because his heart bypass surgeon was incompetent, or because he went right back to eating Big Macs as soon as he was discharged from the hospital?": Keep in mind that we can control for body-mass index, so the question is whether people with similar BMI will be more likely to eat Big Macs after treatment at Hospital A, rather than Hospital B? Yes it's possible, but it seems to me more likely to be an effect of the hospital. Also note that it may not be down to the quality of the heart bypass surgeon - it may be simply that Hospital A is more effective at encouraging healthier eating after patients are discharged. [1] http://www.sts.org/quality-research-patient-safety/sts-public-reporting-online http://www.sts.org/quality-research-patient-safety/sts-publi... [2] https://www.seattlecca.org/sites/default/files/content_page/2016-07/9f/inline-documents/CIBMTR-Final-2015-Transplant-Center-Specific-Survival-Report.pdf https://www.seattlecca.org/sites/default/files/content_page/... [3] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0166762 http://journals.plos.org/plosone/article?id=10.1371/journal.... [4] http://www.newyorker.com/magazine/2004/12/06/the-bell-curve http://www.newyorker.com/magazine/2004/12/06/the-bell-curve