5 ms·

The original paper was an n=46 study, and they have regression equations there with 5 coefficients. It has been known for a long time that you can't successfull

by Gatsky 7y ago

The original paper was an n=46 study, and they have regression equations there with 5 coefficients. It has been known for a long time that you can't successfully adjust for confounding variables with such small data. The fact it didn't reproduce is easily predicted and hardly noteworthy. Everybody in the field is aware of this issue. This article is just a bunch of academics tooting their own horn. The Journal's response to their request was 'Who cares?', and this is the right answer.

michaelterryio 7y ago
Then why was the original study published? And why did it get so much press?
- spamizbad 7y ago
  It was likely published because it was "new". You always need a belt-and-suspenders study if you're sniffing around for interesting phenomena. With that said, often times "Extraordinary claims require extraordinary evidence" as they say. As for why it got lots of press: It's the kind of study that would drive clicks/engagement, increasing ad impressions. So... Capitalism.
WillPostForFood 7y ago
> The Journal's response to their request was 'Who cares?', and this is the right answer. That's what they should have said to the first paper, but since they decided to publish it they can no longer say "who cares" to the correction. They should either retract the original, or publish this.
- bilbo0s 7y ago
  Problem is it's not a "correction". There was nothing really there in the first place, which is what the first paper said to anyone with a modicum of mathematical knowledge. It's like any other paper that reports a statistically non significant result. The appropriate response is, "Well, OK. Sure, whatever." But yeah, we're not gonna take up space from something else that may be important, just to go back and forth over this statistically non significant result. You let them do that in your journal that crap will go on forever and you'll never have space for anything substantive. I'm sure liberal and conservative political types must have some other place they can do that at. If you have a new result in some new area, that's one thing. But if all you have is the exact same result saying that the old non significant result is, um, non significant? I mean, I gotta agree with HN User "Gatsky". "Who cares?" But I'm a research scientist by training and a stickler for protocol. So it's likely my age is showing and the rest of the world wants to loosen up these days? I'm just the old man yelling at the liberal and conservative kids to get off my lawn.
  - Para2016 7y ago
    I care. I bet you still believe Wakefield was right..
deleted 7y ago
[deleted]
sbov 7y ago
So what you're saying is Science publishes studies they know are garbage? Good to know, I'll link your comment anytime someone sends me one of their articles.
- disgruntledphd2 7y ago
  Just don't trust anything published in Science or Nature until it's been replicated by independent researchers. The same is true of all studies, but for some reason people lose their critical faculties when considering work published in the "top" journals.
wdr1 7y ago
Then why publish the original paper?
dagenix 7y ago
They published it. Clearly everyone didn't know that was nonsense or they knowingly published garbage because it would make for nice press and now they refuse to retract it. Either its shameful behavior in the past, or shameful behavior now, or maybe both. If my friend asks me what the score in lasts night's game was, and I tell him 4-3, but then later realize that it was actually 4-2, I'll send him a txt to correct the mistake. If your argument is that a lower standard applies to one of the premier scientific journals, I think that's pretty messed up.
- Gatsky 7y ago
  There is a reproducibility crisis in psychological research. The main reason for this is that the experimental methods they use are not robust (although they might be rigorous). A second reason for this crisis is that the things they are measuring are often hard to actually define (eg is there a definition of 'conservative views' that we can all agree on, and does any such categorization in an experiment really mirror external reality?). In reality what happened with the first paper is the editors of the journal sent it out for review, and whoever reviewed it was happy that it met the methodological standards considered sufficient at the time. This is not the same as being happy that it was true. You might consider that 'messed up' (and you are right, it is messed up), but that was how the sausage was made in psychology research in 2008. There is no galactic council that imposes standards of evidence in a journal. They rely on the reviewers, and the reviewers are active researchers in the field. As an aside, publication in a top journal is not a guarantee that results are water tight, and should never be considered as such. If you want to believe a result, you need to read the paper and think about it carefully. Even then, this approach is susceptible to outright fraud, which also does happen.
  - dagenix 7y ago
    If we have a reproducibility crisis in psychological research, it seems to me like one of the best things we can do to address it is to incentivize people to try reproduce other group's findings. It's not sexy and it won't earn headlines, but it is important. The absolute best way to kill such an effort is for a journal that published a bad study to refuse to even consider publishing a study that refutes it.
tathougies 7y ago
> The Journal's response to their request was 'Who cares?', and this is the right answer. Science cares, clearly!

by Gatsky 7y ago

michaelterryio 7y ago

Then why was the original study published? And why did it get so much press?

spamizbad 7y ago
It was likely published because it was "new". You always need a belt-and-suspenders study if you're sniffing around for interesting phenomena. With that said, often times "Extraordinary claims require extraordinary evidence" as they say. As for why it got lots of press: It's the kind of study that would drive clicks/engagement, increasing ad impressions. So... Capitalism.

WillPostForFood 7y ago

> The Journal's response to their request was 'Who cares?', and this is the right answer. That's what they should have said to the first paper, but since they decided to publish it they can no longer say "who cares" to the correction. They should either retract the original, or publish this.

bilbo0s 7y ago
Problem is it's not a "correction". There was nothing really there in the first place, which is what the first paper said to anyone with a modicum of mathematical knowledge. It's like any other paper that reports a statistically non significant result. The appropriate response is, "Well, OK. Sure, whatever." But yeah, we're not gonna take up space from something else that may be important, just to go back and forth over this statistically non significant result. You let them do that in your journal that crap will go on forever and you'll never have space for anything substantive. I'm sure liberal and conservative political types must have some other place they can do that at. If you have a new result in some new area, that's one thing. But if all you have is the exact same result saying that the old non significant result is, um, non significant? I mean, I gotta agree with HN User "Gatsky". "Who cares?" But I'm a research scientist by training and a stickler for protocol. So it's likely my age is showing and the rest of the world wants to loosen up these days? I'm just the old man yelling at the liberal and conservative kids to get off my lawn.
- Para2016 7y ago
  I care. I bet you still believe Wakefield was right..

deleted 7y ago

[deleted]

sbov 7y ago

So what you're saying is Science publishes studies they know are garbage? Good to know, I'll link your comment anytime someone sends me one of their articles.

disgruntledphd2 7y ago
Just don't trust anything published in Science or Nature until it's been replicated by independent researchers. The same is true of all studies, but for some reason people lose their critical faculties when considering work published in the "top" journals.

wdr1 7y ago

Then why publish the original paper?

dagenix 7y ago

They published it. Clearly everyone didn't know that was nonsense or they knowingly published garbage because it would make for nice press and now they refuse to retract it. Either its shameful behavior in the past, or shameful behavior now, or maybe both. If my friend asks me what the score in lasts night's game was, and I tell him 4-3, but then later realize that it was actually 4-2, I'll send him a txt to correct the mistake. If your argument is that a lower standard applies to one of the premier scientific journals, I think that's pretty messed up.

Gatsky 7y ago
There is a reproducibility crisis in psychological research. The main reason for this is that the experimental methods they use are not robust (although they might be rigorous). A second reason for this crisis is that the things they are measuring are often hard to actually define (eg is there a definition of 'conservative views' that we can all agree on, and does any such categorization in an experiment really mirror external reality?). In reality what happened with the first paper is the editors of the journal sent it out for review, and whoever reviewed it was happy that it met the methodological standards considered sufficient at the time. This is not the same as being happy that it was true. You might consider that 'messed up' (and you are right, it is messed up), but that was how the sausage was made in psychology research in 2008. There is no galactic council that imposes standards of evidence in a journal. They rely on the reviewers, and the reviewers are active researchers in the field. As an aside, publication in a top journal is not a guarantee that results are water tight, and should never be considered as such. If you want to believe a result, you need to read the paper and think about it carefully. Even then, this approach is susceptible to outright fraud, which also does happen.
- dagenix 7y ago
  If we have a reproducibility crisis in psychological research, it seems to me like one of the best things we can do to address it is to incentivize people to try reproduce other group's findings. It's not sexy and it won't earn headlines, but it is important. The absolute best way to kill such an effort is for a journal that published a bad study to refuse to even consider publishing a study that refutes it.

tathougies 7y ago

> The Journal's response to their request was 'Who cares?', and this is the right answer. Science cares, clearly!