3 ms·

Typically papers are reviewed by anonymous peers (other researchers in the field) and the journal's editors. This is why the fraud review board chastised the fi

Typically papers are reviewed by anonymous peers (other researchers in the field) and the journal's editors. This is why the fraud review board chastised the field of Psychology as a whole. Note that it would have been nearly impossible to detect fraud in a single, isolated paper His data were fabricated in such a way as to seem believable, and it's not like researchers are expected to record video evidence of their experiments actually taking place or anything like that; they just report the results. He was ultimately revealed by those who were familiar with his work noticing patterns that appeared over many papers. I don't think there is any formal mechanism in place to review entire bodies of work as a whole.

lmm 14y ago
Missing raw data should surely be a big red flag; maybe not video but surely you're expected to include the individual response numbers so that other people can check your analysis, and it sounds like he claimed to not even have those.
- jorleif 14y ago
  Usually these kinds of data are not part of the publication or up for review. I don't know why exactly, but I suppose it is a combination of: 1) paper publishing tradition, where distributing this material is too much work. 2) too much work for reviewers to comb through raw data, the real solution would be to require sharing of the raw data, so readers rather than the reviewers would find mistakes, but this is often difficult because of privacy issues. E.g. videos of the Utrecht train station study would make fraud more difficult, but publishing the videos would probably require permission from all participants 3) Trust in the authors' integrity. Most scientists would not make up data, but they might make logical errors in their reasoning, or use bad procedures. Peer review can find these errors, but if the authors lie about their procedures, it is very hard to check. Peer review is about internal consistency.

lmm 14y ago

Missing raw data should surely be a big red flag; maybe not video but surely you're expected to include the individual response numbers so that other people can check your analysis, and it sounds like he claimed to not even have those.

jorleif 14y ago
Usually these kinds of data are not part of the publication or up for review. I don't know why exactly, but I suppose it is a combination of: 1) paper publishing tradition, where distributing this material is too much work. 2) too much work for reviewers to comb through raw data, the real solution would be to require sharing of the raw data, so readers rather than the reviewers would find mistakes, but this is often difficult because of privacy issues. E.g. videos of the Utrecht train station study would make fraud more difficult, but publishing the videos would probably require permission from all participants 3) Trust in the authors' integrity. Most scientists would not make up data, but they might make logical errors in their reasoning, or use bad procedures. Peer review can find these errors, but if the authors lie about their procedures, it is very hard to check. Peer review is about internal consistency.