4 ms·
Perhaps relevant to a lot of HN readers - the culture of conference publishing in much of CS (I'm in computer systems myself) seems to result in significantly d
by pjdesno 2y ago
Perhaps relevant to a lot of HN readers - the culture of conference publishing in much of CS (I'm in computer systems myself) seems to result in significantly different dynamics.
Journal papers are single-blind, i.e. the reviewers know the names of the authors and can make decisions based on reputation, so if you publish great (e.g. faked) results once, it becomes progressively easier to do it again.
Reviewing for conference papers is typically double-blind - we don't know the identity of the authors, and although we can try to guess, a couple of studies show we're wrong most of the time. Most papers (65-85% at good conferences) get rejected, so it's easy to flush something that looks fishy. (then again, it's hard to tell the difference between someone playing games with their numbers and someone who just has the typical crappy CS systems understanding of statistics) The addition of artifact evaluation to a lot of conferences provides another avenue to flush out faked results, too.
However conference reviewing is vulnerable to reviewing fraud, since reviewers and authors typically come from the same relatively small community - see https://news.ycombinator.com/item?id=23459978 https://news.ycombinator.com/item?id=23459978, or just google for the words "computer science reviewing cabal".
Reviewing fraud might not be quite as destructive in the long term, as the papers that get published through such fraud are probably more crappy than faked, and will hopefully not get cited. In addition, when reviewing fraud is uncovered it's much more of an open-and-shut case - a single person can fake data and make it look a lot like good data, but review fraud requires communication between people who are explicitly barred from communicating on that topic.
A lot of it depends on the community, and the size of the community. When you have 1000+ reviewers for a conference things will always get out of hand - I know of someone who got his cat registered as a reviewer for a particular SIG conference that he hated, just to make a point. I assume one could get a lot of fraudulent research published in that particular conference.
- raphman 2y agoSometimes, even a whole conference gets compromised, see e.g. ICIMTECH '21 [1]: "NOTICE OF RETRACTION: While investigating potential publication-related misconduct in connection with the ICIMTech 2021 Conference Proceedings, serious concerns were raised that cast doubt on the integrity of the peer-review process and all papers published in the Proceedings of this Conference. The integrity of the entire Conference has been called into question. As a result, of its investigation, ACM has decided to retract the Entire Conference Proceedings and all related papers from the ACM Digital Library. None of the papers from this Proceeding should be cited in the literature because of the questionable integrity of the peer review process for this Conference." [1] https://dl.acm.org/doi/proceedings/10.1145/3465631 https://dl.acm.org/doi/proceedings/10.1145/3465631
- generationP 2y agoOh my, the titles in those proceedings are hilarious. > Retracted on February 24, 2022: Innovation Research on College Student Affairs Management Based on Big Data Environment > Retracted on February 24, 2022: University Students in the Greater Bay Area Conduct Research on the Construction of Big Data Teams for Cross-border E-commerce Startups in the Middle East > Retracted on February 24, 2022: Discussion on International Trade Governance under the Background of Big Data > Retracted on February 24, 2022: Application of Incoterms in International Engineering Based on Information Platform > Retracted on February 24, 2022: The Fixed-point Tracking Model of Volunteer Service for College Students of Traditional Chinese Medicine Based on Information Technology > Retracted on February 24, 2022: Sichuan Consumers' Mining of Agricultural Product Brand Value Based on Big Data > Retracted on February 24, 2022: Sports Consumer Behavior Based on Integrated Data in the Context of Big Data > Retracted on February 24, 2022: Talent Heights Construction from the Perspective of Talent Concentration
- pjdesno 2y agoSounds like they were generated by SCIgen.
- Upvoter33 2y agoI agree about CS reviewing for conferences (esp. in systems) - pretty reasonable for the most part. Also, if things just don't work, they don't work - which is a bit different than studies in social science.