4 ms·

Case size 14, control size 15. It's 2021; you shouldn't be able to publish a psychology result with statistics this weak.

oneoff786 5y ago
That’s an aggressive take for someone who hasn’t bothered to consider the effect size when claiming to know statistics better than these authors.
- carabiner 5y ago
  The heart of the middlebrow dismissal is the assumption that the dismisser understands a field/industry better than a person who's spent their entire adult lives studying it.
rileymat2 5y ago
I am curious what the proper size group is for such dramatic results to make it publishable? The results are quite dramatic, it is not a slim success.
6gvONxR4sf7o 5y ago
Under the null hypothesis, a difference this large is still extremely unlikely for a group this small. Sometimes, counterintuitively, you don’t need a huge group.
- Groxx 5y ago
  Yeah, this is a fairly standard size for an early trial. Trials cost money. Even if this is a lucky outlier and it turns out to be no better than the placebo in a larger trial, having a decently strong positive result can hugely improve your chances of getting the money (and interest) needed to run a larger trial.
  - InfiniteRand 5y ago
    But it’s worth realizing this small of a trial size likely means there’s a long road to widespread use
    - Groxx 5y ago
      Sure, but at that point the complaint is in either the journalism being misleading (implying it's "coming soon" in a layman's use of "coming soon"), or our outsider view into internal messaging is misunderstanding the claims (if this was written for people familiar with the field, "there's a long way to go" is safe to assume as base knowledge, which outsiders may not have). tl;dr yeah, but everyone knows that / science journalism is terrible.
    - carabiner 5y ago
      The same could be said for a massive trial with a small effect.
carabiner 5y ago
Not true. A strong effect can be meaningful for small n. This is well known from power analysis: https://statisticsbyjim.com/hypothesis-testing/sample-size-power-analysis/ https://statisticsbyjim.com/hypothesis-testing/sample-size-p... I also think middlebrow dismissals are discouraged on HN: http://www.byrnehobart.com/blog/why-are-middlebrow-dismissals-so-tempting http://www.byrnehobart.com/blog/why-are-middlebrow-dismissal...

oneoff786 5y ago

That’s an aggressive take for someone who hasn’t bothered to consider the effect size when claiming to know statistics better than these authors.

carabiner 5y ago
The heart of the middlebrow dismissal is the assumption that the dismisser understands a field/industry better than a person who's spent their entire adult lives studying it.

rileymat2 5y ago

I am curious what the proper size group is for such dramatic results to make it publishable? The results are quite dramatic, it is not a slim success.

6gvONxR4sf7o 5y ago

Under the null hypothesis, a difference this large is still extremely unlikely for a group this small. Sometimes, counterintuitively, you don’t need a huge group.

Groxx 5y ago
Yeah, this is a fairly standard size for an early trial. Trials cost money. Even if this is a lucky outlier and it turns out to be no better than the placebo in a larger trial, having a decently strong positive result can hugely improve your chances of getting the money (and interest) needed to run a larger trial.
- InfiniteRand 5y ago
  But it’s worth realizing this small of a trial size likely means there’s a long road to widespread use
  - Groxx 5y ago
    Sure, but at that point the complaint is in either the journalism being misleading (implying it's "coming soon" in a layman's use of "coming soon"), or our outsider view into internal messaging is misunderstanding the claims (if this was written for people familiar with the field, "there's a long way to go" is safe to assume as base knowledge, which outsiders may not have). tl;dr yeah, but everyone knows that / science journalism is terrible.
  - carabiner 5y ago
    The same could be said for a massive trial with a small effect.

carabiner 5y ago

Not true. A strong effect can be meaningful for small n. This is well known from power analysis: https://statisticsbyjim.com/hypothesis-testing/sample-size-power-analysis/ https://statisticsbyjim.com/hypothesis-testing/sample-size-p... I also think middlebrow dismissals are discouraged on HN: http://www.byrnehobart.com/blog/why-are-middlebrow-dismissals-so-tempting http://www.byrnehobart.com/blog/why-are-middlebrow-dismissal...