3 ms·

Can you explain why low percentage improvement over placebo is not important?

• Aurornis 2y ago
  Imagine you have a depression inventory (test) with 21 questions, rated from 0-3. The highest score is 3 * 21 = 63 indicating the most severe depression. The lowest score is 0, indicating no depressive symptoms at all. In practice, the average person will fall more in the range of maybe 5-15 due to vague symptoms like "I don't sleep as well as I used to" triggering some of the points. The average depressed person who seeks treatment might fall in the range of 25-35. Now imagine the placebo group goes in with an average score of 35 and improves to a score of 25 by the end of the test. The SSRI group improves to an average of 20 by the end of the test. Is this significant? Well, it depends on how many patients you have in the sample size. That's the problem. There's only so much room in these scales for improvement, so when both groups improve a lot you need to have a larger sample size to get statistical significance. Getting a lot of patients in a study (hundreds) is very expensive, so it's only a small number of studies that can pull this off.
  • nostrebored 2y ago
    Right, but study power is really the responsibility of a pharma company. It's not like this is some new and novel medication, it's been used for decades and has been questionable for _literally the entirety of the time_.
  • ants_everywhere 2y ago
    The problem with being barely measurably better than a placebo is that each study is a coin flip whether it supports your drug or not. And you can just file drawer any study that didn't go your way (as happens with the majority of null results). So the published results are over-sampling studies where the statistics happened to work and under-sampling studies where they didn't.