4 ms·

> it makes sense to look at change from baseline for individuals in the various groups rather than just differences in raw scores between treatment groups. Tha

> it makes sense to look at change from baseline for individuals in the various groups rather than just differences in raw scores between treatment groups. That’s called cherry picking. Results are always noisy. You could give two groups of people the same tests on different days without any drugs at all and some subset would show “improvement”. If you start focusing on the individuals that show the result you want to see, you cherry-pick your way into false results. This is a well-known way for researchers to abuse variable or noisy data sets to misleadingly show the result they want to show. > I am almost certain that e.g. none of the approved and quite convincingly working SSRI:s would have shown any efficacy in a study with this design and similar N. Thats not true. SSRI studies with ~30 people will show a trend toward improvement in the SSRI group that exceeds the placebo group. I think you’re confusing the different statistical measures. This study showed that expectations and placebo effect were the predictor of micro-dosing success. The blinded group and unblinded group showed completely different results.

zosima 4y ago
No, why would you claim that?
- PragmaticPulp 4y ago
  Claim what? If you go through the results and exclude the individuals who didn’t show the outcome you wanted to see, that’s called cherry-picking. It’s a well-known phrase. There is a concept of subgroup analysis in studies like these, but you have to be careful about how it’s done and what conclusions are drawn. If you simply select positive results and exclude negative results then even the placebo group would show great success. This study showed that telling people that they were microdosing was more important for the perceived outcome than the micro dosing itself. In other words, placebo is key to making it work.
  - _drimzy 4y ago
    > If you simply select positive results and exclude negative results then even the placebo group would show great success. Where did zosima ask to do that? They mentioned that the variable of comparison should be changes from baseline metrics in treatment group vs changes from baseline metrics in control group. That would be a fair study, and isn't cherry picking.
snet0 4y ago
They're not saying you should look at some subset of individuals with positive results, you have misread. The notion is that the difference between participants can mask the effect of the drug, such that comparing any individual participant to anyone but themselves is improper.
karmanyaahm 4y ago
> > it makes sense to look at change from baseline for individuals in the various groups rather than just differences in raw scores between treatment groups. > That’s called cherry picking. Results are always noisy. You could give two groups of people the same tests on different days without any drugs at all and some subset would show “improvement”. No, GP is talking about a matched pairs design. You look at the difference between the scores of very similar individuals by applying one treatment to each (active and placebo), or, applying both treatments to the same individual (in random order). Cherry-picking would mean only using scores from selected individuals, whereas matching only emphasizes the difference.

zosima 4y ago

No, why would you claim that?

PragmaticPulp 4y ago
Claim what? If you go through the results and exclude the individuals who didn’t show the outcome you wanted to see, that’s called cherry-picking. It’s a well-known phrase. There is a concept of subgroup analysis in studies like these, but you have to be careful about how it’s done and what conclusions are drawn. If you simply select positive results and exclude negative results then even the placebo group would show great success. This study showed that telling people that they were microdosing was more important for the perceived outcome than the micro dosing itself. In other words, placebo is key to making it work.
- _drimzy 4y ago
  > If you simply select positive results and exclude negative results then even the placebo group would show great success. Where did zosima ask to do that? They mentioned that the variable of comparison should be changes from baseline metrics in treatment group vs changes from baseline metrics in control group. That would be a fair study, and isn't cherry picking.

snet0 4y ago

They're not saying you should look at some subset of individuals with positive results, you have misread. The notion is that the difference between participants can mask the effect of the drug, such that comparing any individual participant to anyone but themselves is improper.

karmanyaahm 4y ago

> > it makes sense to look at change from baseline for individuals in the various groups rather than just differences in raw scores between treatment groups. > That’s called cherry picking. Results are always noisy. You could give two groups of people the same tests on different days without any drugs at all and some subset would show “improvement”. No, GP is talking about a matched pairs design. You look at the difference between the scores of very similar individuals by applying one treatment to each (active and placebo), or, applying both treatments to the same individual (in random order). Cherry-picking would mean only using scores from selected individuals, whereas matching only emphasizes the difference.