4 ms·

The guy fabricated data to fool the editors. Academia relies on researchers being honest and not doing so -- there simply aren't any resources for journals to d

The guy fabricated data to fool the editors. Academia relies on researchers being honest and not doing so -- there simply aren't any resources for journals to do independent replication of all received papers before accepting them for publication. That's precisely why data fabrication is taken so seriously and the consequences of doing so are basically career ending -- it's the deterrent to incentivize people to be honest. Edit: Also can we not lump all of academia into the same basket? The standards in computer science/math/physics are very different from that in biology/medicine which in turn are very different from those in humanities

microcolonel 8y ago
But the problem with actual data fabrication is that it is intended to permanently deceive, in this case the whole point was to publish the fact that these data were not correct. This is a case where an initial false disclosure (which you seem to think should be taken exactly the same as deliberate, permanent misreporting) is used as a means to bring more verified facts into the world, and zero permanent misinformation; it is no more like false disclosure than pennies are like Moscow Mule mugs.
- letitgo12345 8y ago
  I don't think this is the same as fabrication with intent to commit actual fraud. But if you're trying to show that the editors are making biased decisions because of preconceived notions, submitting data that supports their notions is not evidence of them making biased decisions. A better study would be the following -- you submit one version of the paper supporting viewpoint X and another supporting viewpoint not X (where X is some issue on which there isn't much research been done so there isn't any sort of consensus on the issue) and fabricate data for both cases. But what they did, all that seems to show to me is that if you fabricate data, you can get a paper accepted. Which is not surprising. Edit: For the record, I do think academia, especially in some areas of humanities, is very biased but this study is just not well done imo.
  - microcolonel 8y ago
    > But if you're trying to show that the editors are making biased decisions because of preconceived notions, submitting data that supports their notions is not evidence of them making biased decisions. I think you've missed the point of this. > Edit: For the record, I do think academia, especially in some areas of humanities, is very biased but this study is just not well done imo. How? These papers are basically nonsense, but were accepted because they draw conclusions that are vaguely in line with the political whims of the journals, that's the whole point. One simple question it answers is: would journals publish Mein Kampf if men were "the Jews"? It is a qualitative question, not a quantitative one, so it is sufficient to demonstrate that one such paper is published.
abalone 8y ago
That is like saying The Onion fabricates reporting to fool your aunt on Facebook. These people are just embarrassed they got taken by an obvious work of satire.
- letitgo12345 8y ago
  You have to be open to points of view which may sound ridiculous on the surface though if you can make a case for them. For example, having a model in 1800 that said that everything is both a particle and a wave would have sounded like satire but it wouldn't have been wrong. Or having a point of view in 1800 that interracial relationships are not immoral or in 1900 that gay relationships are not immoral would have sounded like satire but again, not something that would have been the wrong point of view.
  - abalone 8y ago
    These are totally invalid analogies. Scientific claims are never judged by the ridiculousness of the claim but by the strength of the evidence. Every scientific journal would be fairly judged on its assessment of the data and methodology. Every scientific journal is fair game for a “grievance study” and every one that accepts satirically ridiculous methodologies or conclusions would be very fairly judged fraudulent. Your other example confuses popular points of view with academic studies. If there is a better example of the problem here I can’t think of it.
taneq 8y ago
Many of the events depicted in Dilbert aren't based on real research data either. Should Scott Adams be charged with fabricating data?
- letitgo12345 8y ago
  What's your point? Dilbert is a comics strip. Not a research journal where you're making an attempt to further the understanding of an area. If you coded up a networking algorithm and fabricated data showing massive speed improvements and then got a lot of play in the media and networking community before you were found out, would you say that's evidence of the system being broken?
  - jamesrcole 8y ago
    > Not a research journal where you're making an attempt to further the understanding of an area. What if the journal in question is not actually attempting to do that? Might it be moral to try to expose that fact?
  - jabgrabdthrow 8y ago
    Absolutely, isn't that the point!? If the media hypes up a claim based on results that in your example are even supposed to be trivial to replicate!
  - taneq 8y ago
    Maybe if I described such a network using pigeons as the underlying physical layer, and multiple journals took it seriously, then yes.
maxlybbert 8y ago
So if he had actually collected data on rape culture among Portland-area dogs, it would have been legitimate research ("One of their papers, about canine rape culture in dog parks in Portland, Ore., was initially recognized for excellence by the journal 'Gender, Place, and Culture', the authors reported")?
- claudiawerner 8y ago
  If it is relevant data, why shouldn't it be legitimate research? Because you personally think the topic is ridiculous, absurd or not worth consideration? Many ideas seem that way (including very important ones, such as the study of language) so that shouldn't be a reason to dismiss them outright.
  - maxlybbert 8y ago
    I thought people were overstating the case that humanities departments across the country are converged. But if investigating whether dogs actually get consent before mating in dog parks (or is there another definition for "rape culture among dogs"?) is considered possibly legitimate research, I may need to revisit my judgement.
- asdfasgasdgasdg 8y ago
  Probably about as legitimate as the research of obscure computational complexity classes in computer science theory.
- bsder 8y ago
  > So if he had actually collected data on rape culture among Portland-area dogs, it would have been legitimate research Yes, actually. There is a big difference with publishing a paper with no data a la Sokal vs publishing a paper with made up data. If you are presenting data as true, it must be true. Editors and reviewers don't get to argue with your data--that's a cornerstone of the process. They can argue that you collected the wrong data, interpreted it incorrectly, etc. but the data you collected should be real. If this professor really made up data, he absolutely should get fired. I suspect that he probably didn't falsify data or the administration would simply wipe him out fairly unceremoniously. The fact that they are invoking a review board violation tells me that the case isn't that cut and dried.
Jach 8y ago
Only some of the papers fabricated data, most didn't. You can browse the papers here: https://drive.google.com/drive/mobile/folders/19tBy_fVlYIHTxxjuVMFxh4pqLHM_en18 https://drive.google.com/drive/mobile/folders/19tBy_fVlYIHTx...
jules 8y ago
The fabricated data was patently absurd. They claimed that they inspected the genitals of 10,000 dogs at a single dog park. To punish him for this fabrication is ridiculous. They should rather look into the people who reviewed the article and didn't ring the alarm bells immediately after reading such an absurd claim. Data fabrication is taken so seriously because otherwise scientists could game the system by fabricating data to advance their career by making their research look more successful. That obviously didn't apply here -- he didn't even publish the article under his own name.
- deleted 8y ago
  [deleted]
naasking 8y ago
> The guy fabricated data to fool the editors. Academia relies on researchers being honest and not doing so -- there simply aren't any resources for journals to do independent replication of all received papers before accepting them for publication. No, you're simply wrong about this. The publications are not rigourous and they do not perform their due diligence in this field as is done in proper sociological research, and that's what the hoax was intended to demonstrate. I suggest reading my summary here: https://news.ycombinator.com/item?id=18899308 https://news.ycombinator.com/item?id=18899308 This hoax was essential to exposing the poor quality research being done in this field.
- deleted 8y ago
  [deleted]
- waterhouse 8y ago
  I suspect your link should be to https://news.ycombinator.com/item?id=18899308 https://news.ycombinator.com/item?id=18899308 .
  - naasking 8y ago
    Thanks!
buboard 8y ago
> guy fabricated data to fool the editors What is your source for this? My understanding was that the papers were written in a specific style, but they did not contain falsified data. https://areomagazine.com/2018/10/02/academic-grievance-studies-and-the-corruption-of-scholarship/ https://areomagazine.com/2018/10/02/academic-grievance-studi... Regardless, in almost all cases, reviewers do eye-ball the data for obvious inconsistencies / irregularities.
Hasknewbie 8y ago
You are mischaracterizing the research. His submitted papers were not "the data", they were the input. The actual data is how the editors reacted to the input.
mjfl 8y ago
It's okay to fabricate data if you're experiment is to detect whether journal editors and reviewers can pick up on fabricated data.

microcolonel 8y ago

But the problem with actual data fabrication is that it is intended to permanently deceive, in this case the whole point was to publish the fact that these data were not correct. This is a case where an initial false disclosure (which you seem to think should be taken exactly the same as deliberate, permanent misreporting) is used as a means to bring more verified facts into the world, and zero permanent misinformation; it is no more like false disclosure than pennies are like Moscow Mule mugs.

letitgo12345 8y ago
I don't think this is the same as fabrication with intent to commit actual fraud. But if you're trying to show that the editors are making biased decisions because of preconceived notions, submitting data that supports their notions is not evidence of them making biased decisions. A better study would be the following -- you submit one version of the paper supporting viewpoint X and another supporting viewpoint not X (where X is some issue on which there isn't much research been done so there isn't any sort of consensus on the issue) and fabricate data for both cases. But what they did, all that seems to show to me is that if you fabricate data, you can get a paper accepted. Which is not surprising. Edit: For the record, I do think academia, especially in some areas of humanities, is very biased but this study is just not well done imo.
- microcolonel 8y ago
  > But if you're trying to show that the editors are making biased decisions because of preconceived notions, submitting data that supports their notions is not evidence of them making biased decisions. I think you've missed the point of this. > Edit: For the record, I do think academia, especially in some areas of humanities, is very biased but this study is just not well done imo. How? These papers are basically nonsense, but were accepted because they draw conclusions that are vaguely in line with the political whims of the journals, that's the whole point. One simple question it answers is: would journals publish Mein Kampf if men were "the Jews"? It is a qualitative question, not a quantitative one, so it is sufficient to demonstrate that one such paper is published.

abalone 8y ago

That is like saying The Onion fabricates reporting to fool your aunt on Facebook. These people are just embarrassed they got taken by an obvious work of satire.

letitgo12345 8y ago
You have to be open to points of view which may sound ridiculous on the surface though if you can make a case for them. For example, having a model in 1800 that said that everything is both a particle and a wave would have sounded like satire but it wouldn't have been wrong. Or having a point of view in 1800 that interracial relationships are not immoral or in 1900 that gay relationships are not immoral would have sounded like satire but again, not something that would have been the wrong point of view.
- abalone 8y ago
  These are totally invalid analogies. Scientific claims are never judged by the ridiculousness of the claim but by the strength of the evidence. Every scientific journal would be fairly judged on its assessment of the data and methodology. Every scientific journal is fair game for a “grievance study” and every one that accepts satirically ridiculous methodologies or conclusions would be very fairly judged fraudulent. Your other example confuses popular points of view with academic studies. If there is a better example of the problem here I can’t think of it.

taneq 8y ago

Many of the events depicted in Dilbert aren't based on real research data either. Should Scott Adams be charged with fabricating data?

letitgo12345 8y ago
What's your point? Dilbert is a comics strip. Not a research journal where you're making an attempt to further the understanding of an area. If you coded up a networking algorithm and fabricated data showing massive speed improvements and then got a lot of play in the media and networking community before you were found out, would you say that's evidence of the system being broken?
- jamesrcole 8y ago
  > Not a research journal where you're making an attempt to further the understanding of an area. What if the journal in question is not actually attempting to do that? Might it be moral to try to expose that fact?
- jabgrabdthrow 8y ago
  Absolutely, isn't that the point!? If the media hypes up a claim based on results that in your example are even supposed to be trivial to replicate!
- taneq 8y ago
  Maybe if I described such a network using pigeons as the underlying physical layer, and multiple journals took it seriously, then yes.

maxlybbert 8y ago

So if he had actually collected data on rape culture among Portland-area dogs, it would have been legitimate research ("One of their papers, about canine rape culture in dog parks in Portland, Ore., was initially recognized for excellence by the journal 'Gender, Place, and Culture', the authors reported")?

claudiawerner 8y ago
If it is relevant data, why shouldn't it be legitimate research? Because you personally think the topic is ridiculous, absurd or not worth consideration? Many ideas seem that way (including very important ones, such as the study of language) so that shouldn't be a reason to dismiss them outright.
- maxlybbert 8y ago
  I thought people were overstating the case that humanities departments across the country are converged. But if investigating whether dogs actually get consent before mating in dog parks (or is there another definition for "rape culture among dogs"?) is considered possibly legitimate research, I may need to revisit my judgement.
asdfasgasdgasdg 8y ago
Probably about as legitimate as the research of obscure computational complexity classes in computer science theory.
bsder 8y ago
> So if he had actually collected data on rape culture among Portland-area dogs, it would have been legitimate research Yes, actually. There is a big difference with publishing a paper with no data a la Sokal vs publishing a paper with made up data. If you are presenting data as true, it must be true. Editors and reviewers don't get to argue with your data--that's a cornerstone of the process. They can argue that you collected the wrong data, interpreted it incorrectly, etc. but the data you collected should be real. If this professor really made up data, he absolutely should get fired. I suspect that he probably didn't falsify data or the administration would simply wipe him out fairly unceremoniously. The fact that they are invoking a review board violation tells me that the case isn't that cut and dried.

Jach 8y ago

Only some of the papers fabricated data, most didn't. You can browse the papers here: https://drive.google.com/drive/mobile/folders/19tBy_fVlYIHTxxjuVMFxh4pqLHM_en18 https://drive.google.com/drive/mobile/folders/19tBy_fVlYIHTx...

jules 8y ago

The fabricated data was patently absurd. They claimed that they inspected the genitals of 10,000 dogs at a single dog park. To punish him for this fabrication is ridiculous. They should rather look into the people who reviewed the article and didn't ring the alarm bells immediately after reading such an absurd claim. Data fabrication is taken so seriously because otherwise scientists could game the system by fabricating data to advance their career by making their research look more successful. That obviously didn't apply here -- he didn't even publish the article under his own name.

deleted 8y ago
[deleted]

naasking 8y ago

> The guy fabricated data to fool the editors. Academia relies on researchers being honest and not doing so -- there simply aren't any resources for journals to do independent replication of all received papers before accepting them for publication. No, you're simply wrong about this. The publications are not rigourous and they do not perform their due diligence in this field as is done in proper sociological research, and that's what the hoax was intended to demonstrate. I suggest reading my summary here: https://news.ycombinator.com/item?id=18899308 https://news.ycombinator.com/item?id=18899308 This hoax was essential to exposing the poor quality research being done in this field.

deleted 8y ago
[deleted]
waterhouse 8y ago
I suspect your link should be to https://news.ycombinator.com/item?id=18899308 https://news.ycombinator.com/item?id=18899308 .
- naasking 8y ago
  Thanks!

buboard 8y ago

> guy fabricated data to fool the editors What is your source for this? My understanding was that the papers were written in a specific style, but they did not contain falsified data. https://areomagazine.com/2018/10/02/academic-grievance-studies-and-the-corruption-of-scholarship/ https://areomagazine.com/2018/10/02/academic-grievance-studi... Regardless, in almost all cases, reviewers do eye-ball the data for obvious inconsistencies / irregularities.

Hasknewbie 8y ago

You are mischaracterizing the research. His submitted papers were not "the data", they were the input. The actual data is how the editors reacted to the input.

mjfl 8y ago

It's okay to fabricate data if you're experiment is to detect whether journal editors and reviewers can pick up on fabricated data.