4 ms·

Sorry, I will clarify further. There is a difference between statistical and clinical significance. Statistical significance indicates that there is a real eff

Sorry, I will clarify further. There is a difference between statistical and clinical significance. Statistical significance indicates that there is a real effect that is not due to chance. Clinical significance is achieved only if the effect size exceeds a certain threshold. The clinical significance criterion used by the study lutusp linked to is "a three-point difference in Hamilton Rating Scale of Depression (HRSD) scores or a standardized mean difference (d) of 0.5." That study shows antidepressant treatment does not have clinically significant effects beyond placebo in all but the most severe cases of depression; the effect size is quite small. However, the effects are statistically significant (in previous meta-analyses of published studies and this meta-analysis of both published and unpublished studies) and thus likely to be real, and they are clinically significant in the most severe cases of severe depression, even when unpublished studies are included. To summarize, antidepressants do work: their effect is statistically significant. But they don't work very well: their effect is not clinically significant in any but the most severe cases.

dllthomas 12y ago
"However, the effects are statistically significant and thus likely to be real (whether based on published or both published and unpublished studies)" I don't see how you get the parenthesized bit.
- simonster 12y ago
  I put the parens in the wrong place. What I meant was that the effects are statistically significant in both previous meta-analyses and this one. I've adjusted my comment above to make it a bit clearer.
  - dllthomas 12y ago
    Ah, that clarifies quite a bit.
lutusp 12y ago
> To summarize, antidepressants do work: their effect is statistically significant. This is deliberately misleading. You just explained that statistical significance is not clinical significance, but saying "antidepressants do work" strongly implies that they subjectively improve a person's state. These are words you chose, words that contradict the study's conclusion with respect to human subjects in all but the most severely depressed. > But they don't work very well: their effect is not clinically significant in any but the most severe cases. This is like saying, "the skydiver was perfectly all right until he hit the ground." It's misleading. If he used language like this, a clinician could be accused of unethically misleading his patients. The reason for the wide gap between statistical and clinical significance is to guard against the distorting effect of self-reporting, always a risk in a study like this. I want to repeat what you said above, but broken down: (a) "To summarize, antidepressants do work:" (b) "their effect is statistically significant." But according to your own analysis, item (a) does not -- cannot -- acquire its authority from item (b), for the reason that statistical significance doesn't lead to the claim that these drugs "work" in the commonly accepted sense, i.e. subjective improvement in a human subject. I refer again to the difference between science and pseudoscience I quoted earlier -- pseudoscientists seek confirmation, scientists seek falsification. The outcome for the layman? If a scientist cannot falsify something, it acquires a small bit of temporary credibility, but if a pseudoscientist says something works, well, read the study behind the words.

dllthomas 12y ago

"However, the effects are statistically significant and thus likely to be real (whether based on published or both published and unpublished studies)" I don't see how you get the parenthesized bit.

simonster 12y ago
I put the parens in the wrong place. What I meant was that the effects are statistically significant in both previous meta-analyses and this one. I've adjusted my comment above to make it a bit clearer.
- dllthomas 12y ago
  Ah, that clarifies quite a bit.

lutusp 12y ago

> To summarize, antidepressants do work: their effect is statistically significant. This is deliberately misleading. You just explained that statistical significance is not clinical significance, but saying "antidepressants do work" strongly implies that they subjectively improve a person's state. These are words you chose, words that contradict the study's conclusion with respect to human subjects in all but the most severely depressed. > But they don't work very well: their effect is not clinically significant in any but the most severe cases. This is like saying, "the skydiver was perfectly all right until he hit the ground." It's misleading. If he used language like this, a clinician could be accused of unethically misleading his patients. The reason for the wide gap between statistical and clinical significance is to guard against the distorting effect of self-reporting, always a risk in a study like this. I want to repeat what you said above, but broken down: (a) "To summarize, antidepressants do work:" (b) "their effect is statistically significant." But according to your own analysis, item (a) does not -- cannot -- acquire its authority from item (b), for the reason that statistical significance doesn't lead to the claim that these drugs "work" in the commonly accepted sense, i.e. subjective improvement in a human subject. I refer again to the difference between science and pseudoscience I quoted earlier -- pseudoscientists seek confirmation, scientists seek falsification. The outcome for the layman? If a scientist cannot falsify something, it acquires a small bit of temporary credibility, but if a pseudoscientist says something works, well, read the study behind the words.