4 ms·
Hmmm. Here we see the following claim: > By partaking in a form of fraud that has left the Overton window of acceptability, the researchers in the collusion r
by thaumasiotes 5y ago
Hmmm.
Here we see the following claim:
> By partaking in a form of fraud that has left the Overton window of acceptability, the researchers in the collusion ring have finally succeeded in forcing the community to acknowledge its blind spot. For the first time, researchers reading conference proceedings will be forced to wonder: does this work truly merit my attention? Or is its publication simply the result of fraud?
But I don't see how this follows. If I follow the link to the description of the actual fraud ( https://cacm.acm.org/magazines/2021/6/252840-collusion-rings-threaten-the-integrity-of-computer-science-research/fulltext#FNA https://cacm.acm.org/magazines/2021/6/252840-collusion-rings... ), it says essentially the opposite: the "fraudulent" papers are no different from papers published by ordinary means.
> the review process is notoriously random. In a well-publicized case in 2014, organizers of the Neural Information Processing Systems Conference formed two independent program committees and had 10% of submissions reviewed by both. The result was that almost 60% of papers accepted by one program committee were rejected by the other, suggesting that the fate of many papers is determined by the specifics of the reviewers selected
> In response, some authors have adopted paper-quality-independent interventions to increase their odds of getting papers accepted. That is, they are cheating.
> Here is an account of one type of cheating that I am aware of: a collusion ring.
> A group of colluding authors writes and submits papers to the conference.
> The colluders share, amongst themselves, the titles of each other's papers, violating the tenet of blind reviewing
> The colluders hide conflicts of interest, then bid to review these papers, sometimes from duplicate accounts, in an attempt to be assigned to these papers as reviewers.
> The colluders write very positive reviews of these papers
So the system is: conferences already can't tell the difference between a good paper and a bad paper. Researchers respond by adopting strategies for passing review that are irrelevant to paper quality (since paper quality doesn't count). But those strategies aren't bad for paper quality. If I'm reading conference papers, why would I worry about whether one of them is the product of review collusion?
- tom_mellior 5y ago> If I'm reading conference papers, why would I worry about whether one of them is the product of review collusion? Because the one you are reading may have crowded out a better one. Even if the current review system is essentially random, replacing it with something that is essentially a contest of well-connectedness is worse. Young researchers with good ideas but fewer connections, or people from less well-known institutions would have their ideas suppressed. So you should be worrying about stagnation, and about not reading what might actually be new and exciting.
- thaumasiotes 5y agoNone of that reflects negatively on the paper. There is no additional caution warranted when reading papers. That's just a question of "are you happy with the state of the world?". You can think about that question any time.
- YeGoblynQueenne 5y ago>> None of that reflects negatively on the paper. I think it does. Good papers are written by people who care about doing good research, whose primary motivation is to do good research. Such people will not accept to collude or commit academic fraud to get their papers published, because they are idealistic and are in research because they want to do useful work. To such people, committing fraud is anathema, for personal reasons that have nothing to do with economic or other incentives. There are such people in academia but they are also crowded out, to borrow tom_mellior's turn of phrase, from others, who don't hesitate to commit academic fraud to get published and who don't give two flying figs about the quality of their own work. This is obviously a concerning state of affairs that can only be detrimental to the overall quality of research. So I'm sorry but you're dismissing the issue out of hand without having thought about all the consequences. Academic fraud is like, I don't know, broken windows? It just perverts everything around it and creates a black hole of bullshit that sucks everything in it. Good research cannot thrive in such conditions.
- YeGoblynQueenne 5y agoActually, today an article was posted to HN (not by me) that points to Basquiat's parable of the broken window: The parable of the broken window was introduced by French economist Frédéric Bastiat in his 1850 essay "Ce qu'on voit et ce qu'on ne voit pas" ("That Which We See and That Which We Do Not See") to illustrate why destruction, and the money spent to recover from destruction, is not actually a net benefit to society. https://en.wikipedia.org/wiki/Parable_of_the_broken_window https://en.wikipedia.org/wiki/Parable_of_the_broken_window I think this is relevant to our discussion, in particular the concept of "opportunity costs": https://en.wikipedia.org/wiki/Opportunity_cost https://en.wikipedia.org/wiki/Opportunity_cost In microeconomic theory, opportunity cost is the loss of the benefit that could have been enjoyed if the best alternative choice was chosen instead.[1]
- deleted 5y ago[deleted]
- anonymousDan 5y agoYa the guy has barely read or understood the article he is basing all his complaints on.
- deleted 5y ago[deleted]
- user-the-name 5y agoThe author is saying that all of the papers are bullshit, garbage, and a waste of time. He is saying that everyone is turning a blind eye to this. He is saying that now that the fraud has got this bad, it is out in the open, and people will be forced to deal with the fact that the entire field is filled with garbage.
- YeGoblynQueenne 5y agoThat's right. Btw, I think the author is wrong. This is not "out in the open" now. It's been "out in the open" since for ever. And those who act all surprised about it are probably the ones who knew it better than most. I don't mean the author, of course, but he is pointing to some of his own papers as bullshit. Why did he publish bullshit papers if he knew they were bullshit? Or doese he mean those papers are bullshit with hindsight? That, I can understand- "I wrote this paper ten years ago and reading it now, I cringe". Sure, that happens and it's only evidence of the person's progress. But to say that one's research was bullshit in the context of an article that describes academic fraud that leads to the publication of bullshit? That I don't get. Surely the author is not confessing to committing academic fraud himself! I didn't get that from the footnote anyway.
- jacobbuckman 5y agoWhen I published those papers, I was new to the field. I was guided by the standards set by the community, by my mentors, by my peers. I was proud of each of those papers. At publication time, I believed I was doing good science, and the belief was re-affirmed by acceptance at top conferences. My thinking has evolved since then. The community norms have not.
- YeGoblynQueenne 5y agoOh, er, sorry for the harsh words, anyway I hope they didn't come across as too harsh. So I think you're saying the papers you linked to where "bullshit with hindsight", not that they were unethical at the time. Or at least you didn't think they were.
- YeGoblynQueenne 5y ago>> If I'm reading conference papers, why would I worry about whether one of them is the product of review collusion? Well, because if there's no incentive to write a good paper, very few good papers will ever get written and you'll waste your time reading crap papers. I get what you say: if the process is random anyway, then what's the problem? But the process is not random: authors maximise their chances to publish their paper if they partake in a collusion ring. Partaking in a collusion ring diverts resources from writing a good paper, therefore the quality of published papers goes down the drain. The most important resource necessary to write a good paper of course is the motivation to not publish crap. If that goes, everything else follows.