3 ms·

>"To return to the HAART example, imagine you're an HIV+ patient and I've told you that a major study failed to control for time-varying confounding, and that u

by nonbel 9y ago

>"To return to the HAART example, imagine you're an HIV+ patient and I've told you that a major study failed to control for time-varying confounding, and that upon re-analysis, instead of doubling your 5 year survival, it only increases it by 87%. Or, for a form of "is this repeatable?" that I particularly despise, that it still doubles your chances of survival, but the p-value has gone from 0.047 to 0.062. Do you want to stop taking the drug?" This obviously depends on the relative costs. Cost of side effects, buying the drugs, time going to treatment, etc. That all requires accurate quantification. But cost-benefit isn't even what I meant. To begin with, if they can't nail down a quantifiable effect that is stable from study to study, who knows what is going on? Why would you have confidence in their estimates of effectiveness if they are inconsistent with one another?

Fomite 9y ago
"This obviously depends on the relative costs. Cost of side effects, buying the drugs, time going to treatment, etc." I'm going to suggest if you're facing death from an AIDS-related illness, a relative risk of 2.00 vs. 1.87 will feel very, very similar to you. "To begin with, if they can't nail down a quantifiable effect that is stable from study to study, who knows what is going on? Why would you have confidence in their estimates of effectiveness if they are inconsistent with one another?" Define "stable" - because even if the effect of something is fixed in the same way a physical constant it, it's invariable sampled with error. Again, if I give you five studies that suggest that HAART improves survival by: 100%, 102%, 87%, 94% and 96% are you really going to suggest that we don't know that HAART improves survival?
- nonbel 9y ago
  >"I'm going to suggest if you're facing death from an AIDS-related illness, a relative risk of 2.00 vs. 1.87 will feel very, very similar to you." I at first thought we both understood there would be some uncertainty about these values no matter what but we were leaving that out out for simplicity's sake. Roughly, I was assuming it is at least +/- 10-20%. The reason these values "feel very, very similar" is that I would not expect medical data to be able to distinguish between them. >"Define "stable" - because even if the effect of something is fixed in the same way a physical constant it, it's invariable sampled with error." Ok... so you are implicitly considering uncertainty. >"Again, if I give you five studies that suggest that HAART improves survival by: 100%, 102%, 87%, 94% and 96% are you really going to suggest that we don't know that HAART improves survival?" Not enough info. - Where did the uncertainty go? - What methods were used to generate these values? Even basic strategies like blinding the people collecting/processing/analyzing data is often still missing. - What population/frame do those numbers refer to, and how much error can we expect from extrapolating for other situations in the future? You need a reliable quantification, doing this hand wavy "things are better/worse, significant/insignificant" is an awful idea.

by nonbel 9y ago

Fomite 9y ago

"This obviously depends on the relative costs. Cost of side effects, buying the drugs, time going to treatment, etc." I'm going to suggest if you're facing death from an AIDS-related illness, a relative risk of 2.00 vs. 1.87 will feel very, very similar to you. "To begin with, if they can't nail down a quantifiable effect that is stable from study to study, who knows what is going on? Why would you have confidence in their estimates of effectiveness if they are inconsistent with one another?" Define "stable" - because even if the effect of something is fixed in the same way a physical constant it, it's invariable sampled with error. Again, if I give you five studies that suggest that HAART improves survival by: 100%, 102%, 87%, 94% and 96% are you really going to suggest that we don't know that HAART improves survival?

nonbel 9y ago
>"I'm going to suggest if you're facing death from an AIDS-related illness, a relative risk of 2.00 vs. 1.87 will feel very, very similar to you." I at first thought we both understood there would be some uncertainty about these values no matter what but we were leaving that out out for simplicity's sake. Roughly, I was assuming it is at least +/- 10-20%. The reason these values "feel very, very similar" is that I would not expect medical data to be able to distinguish between them. >"Define "stable" - because even if the effect of something is fixed in the same way a physical constant it, it's invariable sampled with error." Ok... so you are implicitly considering uncertainty. >"Again, if I give you five studies that suggest that HAART improves survival by: 100%, 102%, 87%, 94% and 96% are you really going to suggest that we don't know that HAART improves survival?" Not enough info. - Where did the uncertainty go? - What methods were used to generate these values? Even basic strategies like blinding the people collecting/processing/analyzing data is often still missing. - What population/frame do those numbers refer to, and how much error can we expect from extrapolating for other situations in the future? You need a reliable quantification, doing this hand wavy "things are better/worse, significant/insignificant" is an awful idea.