4 ms·

"Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian sha

"Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian shape." In other words, researchers cut corners. You should never assume that something is a certain way without rigorously proving it. How did these papers make it past peer review?

Vexs 10y ago
They were probably peer-reviewed by biologists, not statisticians. Peer review is mostly just a good think and read for about a month, not a perfect analysis.
- jessriedel 10y ago
  ...as long as by "read for about a month" you mean "sit on it for a month, then read for a few hours".
  - leemailll 10y ago
    haha, so true
trentmb 10y ago
n=30 ought to be enough for anybody
- leemailll 10y ago
  n>=30 is not the same as gaussian distribution
  - trentmb 10y ago
    Correct.
- psycr 10y ago
  Depends how big the effect size is. For putting a loaded revolver next to your head, and pulling the trigger, n=30 is plenty.
  - trentmb 10y ago
    Correct.
acveilleux 10y ago
fMRI papers tend to be written by neuro-psychologists or similar folks. Not engineers, statisticians, or physicists. They get reviewed by them too.
leemailll 10y ago
Peer-review in biology-related fields doesn't bother too much about the details of your code. Only recently a few journals begin to ask guys to submit the source code if the manuscript is mainly about some new analysis methods. So it is more than likely even if you are reading submitted manuscript of a computer program, you won't read the source code with the paper. Another thing is that statistical analysis is kind of the missing piece in many researches training. It maybe not be "researchers cut corners", but they might not know some of the prerequirements for the analyses used. Even in this year, 2016 when I talked with a colleague in my institute I found out he didn't know a single assumption for t-test, who is a senior postdoc and used it everyday.
marmaduke 10y ago
Developing a statistical threshold requires some null hypothesis, which very often takes Gaussian or linear form. The paper on which the method is based likely states their assumptions clearly, allowing their result to pass peer review. Papers merely employing some method are rarely reviewed by the same peers as those of the methods papers, and often neither aware of or state the assumptions of the method. This is how this happens, at least in neuroscience. You can look down on it and judge, but neuroscience has not really found its Kuhnian paradigm yet, so be nice.
thaw13579 10y ago
I reckon there is a vast number of similar problems in other studies across most fields. Linear regression, ANOVA, and t-tests are widely used techniques with an assumption of Gaussian errors that is rarely checked. I wonder how many of these papers would become null results if switched to nonparametric tests...

Vexs 10y ago

They were probably peer-reviewed by biologists, not statisticians. Peer review is mostly just a good think and read for about a month, not a perfect analysis.

jessriedel 10y ago
...as long as by "read for about a month" you mean "sit on it for a month, then read for a few hours".
- leemailll 10y ago
  haha, so true

trentmb 10y ago

n=30 ought to be enough for anybody

leemailll 10y ago
n>=30 is not the same as gaussian distribution
- trentmb 10y ago
  Correct.
psycr 10y ago
Depends how big the effect size is. For putting a loaded revolver next to your head, and pulling the trigger, n=30 is plenty.
- trentmb 10y ago
  Correct.

acveilleux 10y ago

fMRI papers tend to be written by neuro-psychologists or similar folks. Not engineers, statisticians, or physicists. They get reviewed by them too.

leemailll 10y ago

Peer-review in biology-related fields doesn't bother too much about the details of your code. Only recently a few journals begin to ask guys to submit the source code if the manuscript is mainly about some new analysis methods. So it is more than likely even if you are reading submitted manuscript of a computer program, you won't read the source code with the paper. Another thing is that statistical analysis is kind of the missing piece in many researches training. It maybe not be "researchers cut corners", but they might not know some of the prerequirements for the analyses used. Even in this year, 2016 when I talked with a colleague in my institute I found out he didn't know a single assumption for t-test, who is a senior postdoc and used it everyday.

marmaduke 10y ago

Developing a statistical threshold requires some null hypothesis, which very often takes Gaussian or linear form. The paper on which the method is based likely states their assumptions clearly, allowing their result to pass peer review. Papers merely employing some method are rarely reviewed by the same peers as those of the methods papers, and often neither aware of or state the assumptions of the method. This is how this happens, at least in neuroscience. You can look down on it and judge, but neuroscience has not really found its Kuhnian paradigm yet, so be nice.

thaw13579 10y ago

I reckon there is a vast number of similar problems in other studies across most fields. Linear regression, ANOVA, and t-tests are widely used techniques with an assumption of Gaussian errors that is rarely checked. I wonder how many of these papers would become null results if switched to nonparametric tests...