21 ms·

Please commit more blatant academic fraud

• igorkraw 5y ago
  I've met Jacob in person and we had a super interesting discussion about RL that sadly petered out during the covid lockdown. This blog post very much fits my impression of him and I give mad probs to shitting on your own work (particularly the thermometer encoding was a pet peeve of mine).
• blagie 5y ago
  It's worth noting: In most of these cases, even when caught, the guilty parties and the bad papers remain unnamed. There are known-to-the-editors fraudulent papers in major journals still being cited. The consequences of committing academic fraud are minor, and the consequences of not doing so generally mean no academic jobs or tenure. There isn't a 99.99% innocent claim here (as at the end of the article). This stuff is widespread. This is much more accurate: "because everybody is complicit in this subtle fraud, nobody is willing to acknowledge its existence" Littman is at Brown, where the majority of the CS department engages in this !@#$%. I'm not trying to single out Brown. Parts of MIT, Stanford, etc. are even worse.
  • MattGaiser 5y ago
    With laws, there are full time police officers, investigators, and prosecutors working to catch people. And even then, most crimes go unsolved and unpunished. Even without the complicity, stopping fraud and bad behavior is a difficult problem. Part time reviewers and reputation seem woefully insufficient counters.
    • qPM9l3XJrF 5y ago
      Yes, I'm very skeptical of Buckman's claim that more fraud will help. Seems similar to a claim around 2015-2016 that voting for Donald Trump will inspire America to clean up its act politically. Institutions aren't biological systems that operate according to mysterious hormetic processes. Institutions are created by humans and thrive or decay based on whether they effectively channel human effort. Be the change you want to see in the world. And if you wreck something, and others apply blood sweat and tears to recreate it in a way that's better than the original, you don't get to take any credit as the wrecker.
      • MattGaiser 5y ago
        I suspect that we see terrible events as changing things for the better because when something major happens, we notice a strong reaction. We forget all the not quite as awful things that were just let slide and normalized.
      • blagie 5y ago
        I think the overwhelming problem is the amount of trust people have an academia, and especially, in elite academia. A Harvard research paper is trusted. The harm of people trusting fake science is high. I see it every day in my field. 1) If people realize it's more like a Facebook post, that will be better. People will be able to push back. 2) Alternatively, if we clean up this mess, that will be even better. Academia ought to be trustworthy. I don't see a path to #2 without a lot of dishonest people with tenure being laid off. At elite institutions, most people hired in the past decade or two cheated at least a little bit. I don't see a path to get there without a high-profile scandal.
    • FabHK 5y ago
      That would be interesting, if universities had full time academic integrity officers whose job was to detect academic fraud.
      • MattGaiser 5y ago
        Or just have some academic equivalent to SDETs. Have a replication and analysis team that checks that statistical quality, code, basic science, etc.
      • immmmmm 5y ago
        after having spent a third of my life in academia i don't see this happening: everyone want as many as possible articles, if possible in the most high impact journals. detecting fraudulent or irrelevant pieces is against that goal.
      • matthewdgreen 5y ago
        I don't think people understand how hard this would be, or how much it would impede science. To give an analogy to coding: imagine a company with thousands of programmers, all using different languages and writing different types of code (mobile apps, back-end server apps, mainframe code, device firmware). Now assign one or two poorly-paid employees whose job is to review all of their code and find issues with it. Can you imagine how annoying this would be? How much time you would spend simply explaining to these people what your code does, so they could understand the basics of what your program is even doing? How ineffective they would be at detecting actual fraud, and how quickly their processes would turn into yet another annoying layer of bureaucracy and checklist compliance? There are a lot of things Universities could do to assist scientists in producing better output. This is not one of them.
• seesawtron 5y ago
  Previous thread: https://news.ycombinator.com/item?id=27295906 https://news.ycombinator.com/item?id=27295906
• thaumasiotes 5y ago
  Hmmm. Here we see the following claim: > By partaking in a form of fraud that has left the Overton window of acceptability, the researchers in the collusion ring have finally succeeded in forcing the community to acknowledge its blind spot. For the first time, researchers reading conference proceedings will be forced to wonder: does this work truly merit my attention? Or is its publication simply the result of fraud? But I don't see how this follows. If I follow the link to the description of the actual fraud ( https://cacm.acm.org/magazines/2021/6/252840-collusion-rings-threaten-the-integrity-of-computer-science-research/fulltext#FNA https://cacm.acm.org/magazines/2021/6/252840-collusion-rings... ), it says essentially the opposite: the "fraudulent" papers are no different from papers published by ordinary means. > the review process is notoriously random. In a well-publicized case in 2014, organizers of the Neural Information Processing Systems Conference formed two independent program committees and had 10% of submissions reviewed by both. The result was that almost 60% of papers accepted by one program committee were rejected by the other, suggesting that the fate of many papers is determined by the specifics of the reviewers selected > In response, some authors have adopted paper-quality-independent interventions to increase their odds of getting papers accepted. That is, they are cheating. > Here is an account of one type of cheating that I am aware of: a collusion ring. > A group of colluding authors writes and submits papers to the conference. > The colluders share, amongst themselves, the titles of each other's papers, violating the tenet of blind reviewing > The colluders hide conflicts of interest, then bid to review these papers, sometimes from duplicate accounts, in an attempt to be assigned to these papers as reviewers. > The colluders write very positive reviews of these papers So the system is: conferences already can't tell the difference between a good paper and a bad paper. Researchers respond by adopting strategies for passing review that are irrelevant to paper quality (since paper quality doesn't count). But those strategies aren't bad for paper quality. If I'm reading conference papers, why would I worry about whether one of them is the product of review collusion?
  • tom_mellior 5y ago
    > If I'm reading conference papers, why would I worry about whether one of them is the product of review collusion? Because the one you are reading may have crowded out a better one. Even if the current review system is essentially random, replacing it with something that is essentially a contest of well-connectedness is worse. Young researchers with good ideas but fewer connections, or people from less well-known institutions would have their ideas suppressed. So you should be worrying about stagnation, and about not reading what might actually be new and exciting.
• 2sk21 5y ago
  Thank goodness I am retired and don't have to publish stuff in the current environment. Thirty years ago when machine learning was a sleepy backwater, life was a lot easier for us.
  • sillysaurusx 5y ago
    As a present-day researcher (or at least someone who aspires to be), I'm eagerly waiting for the hype to die down, and for ML to become a sleepy backwater again. You were lucky to play around with ML during those days. The neat thing is, the next AI winter, we'll still have massive hardware rigs. That's very different from 30 years ago. It's a lot easier to discover new things when you can just test every possibility overnight, rather than carefully planning. So if "winter is coming," it will be a lot less harsh this time.
    • busterarm 5y ago
      That kind of compute still costs money. You still have to find the funding for it.
      • sillysaurusx 5y ago
        Nah, a motivated hacker can use a Colab TPU or GPU for free. It's how I started. There's an ungodly amount of resources available now compared to even 10 years ago, let alone 30. The bottleneck is usually motivation.
        • YeGoblynQueenne 5y ago
          Note that all the advances that eventually led to today's successes and applications of deep learning happened 20 or more years ago (e.g. LSTMs was in the '90s, the neocognitron in the '80s and so on). Then the hardware and the data came online and the field boomed, but no progress has been made in terms of new approaches and the overall state of research has stagnated with the inevitable results discussed in the article above (I can quote at least one Turing Award winner on that if that sounds just like me bloviating). In a sense, because it's now easy enough to get good results by throwing a bunch of data at a large computer, everyone's at it and nobody is looking for a way to obtain good results without a lot of data and a big computer. Which of course is not sustainable, not least because it means only large corporations can obtain state of the art results (and to be honest, I don't think any such have been obtained by "motivated hackers" using colab). No, my hope is that things will continue as they are now, until academics realise they've been kicked out of the game by the Big Players, and then go look for a way to compete that doesn't require gigantic amounts of compute. After all, academia can still motivate people to do actual research and find actually new things. The same motivation is much harder in industry, that only cares about one thing and only knows how to hand out one kind of reward.
• deleted 5y ago
  [deleted]
• berndi 5y ago
  Wow, the amount of depravity in some academic circles is astonishing. The article links [1] the case of a PhD student at the University of Florida who was forced to participate in such a publication collusion ring and was pressured to commit scientific fraud by fabricating results and submitting them to a conference [2], being threatened with physical harm should he decide to go public. This student saw no way out and decided to kill himself. Just a few days after the suicide, the department thought it would be appropriate for the student's own lab to have a "fun" excursion and to document it on Instagram [3]. I'm lost for words. [1] https://medium.com/@tnvijayk/potential-organized-fraud-in-acm-ieee-computer-architecture-conferences-ccd61169370d https://medium.com/@tnvijayk/potential-organized-fraud-in-ac... [2] https://huixiangvoice.medium.com/the-hidden-story-behind-the-suicide-phd-candidate-huixiang-chen-236cd39f79d3 https://huixiangvoice.medium.com/the-hidden-story-behind-the... [3] https://www.instagram.com/p/Bz1EExLhdYD/ https://www.instagram.com/p/Bz1EExLhdYD/
  • jollybean 5y ago
    Imagine if this happened in private enterprise, like at Google. The press would be all over it 24/7.
    • totalZero 5y ago
      Yes, I agree. It seems to me that academia feels external to the power structure, so it rarely gets the same kind of attention as industry because it doesn't stir up resentment.
      • jollybean 5y ago
        They are part of the power structure, they're just 'protected'. Most publications don't want to be seen as promoting a narrative that goes 'against science' even though of course it's not. There's no room for nuance in populism.
        • 3grdlurker 5y ago
          Protected, or just less noticed.
• intricatedetail 5y ago
  Scientific community tainted itself by producing fake medical cannabis "research" as they could only get funding if their "studies" could prove negative effects.
  • lowdose 5y ago
    Out of curiosity could you refer me to some sources to read more about this.
    • intricatedetail 5y ago
      https://www.greenentrepreneur.com/article/357200 https://www.greenentrepreneur.com/article/357200 This article scratches the surface.
• sillysaurusx 5y ago
  One surprising discovery about AI: There's nothing inherently wrong with cherrypicking. I used to think it was this dirty thing. Oh, you're only showing your best results? But the best results are what matters. If you can create an amazing song from scratch by telling a computer what to do, you only need to do it once. The song is still good. And it's easy to automate. If you have a way of detecting a good song, or at least filtering out rubbish, then you can generate thousands of attempts to get the good result. I like to say "It's hard to pick cherries from a rotten tree." The flipside of this post, though, really hits home. I recently was super excited about a paper called FNet, which showed that "fourier transforms can be competitive with transformers", i.e. just replace multihead attention with a fourier transform. 7x faster on a GPU! Woo! Buried in section 3.1, they casually note that BERT with half the parameter count completely dominates them in terms of accuracy: https://twitter.com/theshawwn/status/1393315603973386240 https://twitter.com/theshawwn/status/1393315603973386240 In other words, the paper was ... advertised in a misleading way, to put it kindly. Most of the transformers they compared with aren't the traditional multihead GPT style transformer that people think of when they hear "transformer". And the one that was (BERT) totally annihilated them. But it's also easy to jump to conclusions too quickly. I've been a hothead in the past, and called out a paper when it turned out that I just didn't understand enough. It's tricky to know for sure.
  • teruakohatu 5y ago
    > . If you have a way of detecting a good song, or at least filtering out rubbish, then you can generate thousands of attempts to get the good result. So often the filtering technique is a human in the loop, manually going through the output. Humans become the hidden, undifferentiatable and undisclosed loss function.
    • sillysaurusx 5y ago
      Sure is! And you'd often be amazed at just how much data you can churn through, if you put your mind to it. I once watched Gwern classify thousands of poetry outputs over the course of a day or two, with nothing more than dedication and a bash script to mark "a" or "b". It took a long time to accept that this is both normal and effective. It's not some hidden secret to be ashamed of. The headline images in https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/ are probably the best and most interesting images they could find; they were picked out by humans. It's a positive statement to say that dall-e can do that some of the time, rather than "it fails X% of the time."
• wincent 5y ago
  Best part of the article is the footnote, where the author characterizes his own papers: > This paper is bullshit, this paper (a NeurIPS oral) is bullshit, this paper is complete bullshit, this paper is mostly good science but also has a sprinkling of bullshit. Apologies to my co-authors. https://jacobbuckman.com/2021-05-29-please-commit-more-blatant-academic-fraud/#fn:0 https://jacobbuckman.com/2021-05-29-please-commit-more-blata...
  • lixtra 5y ago
    The author already gave it away in the abstract, though. Up to two digits is a good indicator of bullshit (by both author and review). [1]: >> score of up to 93.30% [1] https://www.aclweb.org/anthology/D16-1254.pdf https://www.aclweb.org/anthology/D16-1254.pdf
• guilhas 5y ago
  Trust the science
• vinni2 5y ago
  Which is why several journals are pushing for reproducibility as a requirement and dedicated reproducibility tracks where there are opportunities to call bluff on established papers. As an academic from a small lab in an unknown university it is shocking for me when we are unable to reproduce performance numbers from papers by popular institutions.
  • bonoboTP 5y ago
    Reproducibility doesn't begin to solve the subtler issues mentioned in the article. It is necessary but far from sufficient.
  • teachingassist 5y ago
    > several journals are pushing for reproducibility as a requirement The current requirement for academic publishing is peer-review, i.e.: "Colleagues in my field looked over it briefly and said it's ok [because it doesn't challenge their work]" This inevitably leads to whole fields of science being corrupted and biased in ways that we can't easily identify. Science publishing standards could be improved in any number of ways: e.g. original data must be shared in an open format, publish your hypotheses before doing the research, publish the code, standardise the setting out of methods and statistics and results so that other scientists in and out of the field have a hope of following them. Nobody in the industry is motivated to change this, and hides behind the romantic ideal of a scientist pushing the boundaries and breaking the rules to discover novel theories. If such a scientist exists today, we can't identify them against the background of noise.
    • alisonkisk 5y ago
      Modern science doesn't have peer review! Peer review works within a community, scientists review each other's work to decide who to trust. But the public and funding agencies are not part of that review! So why to we believe or fund the results? Either we are not their peers and should not engage with them, or we should be reviewing their work as part of the publication process. We also have "peers" in criminal prosecution. Would you allow a criminal trial where only the scientist's colleagues were in the jury?
• bonoboTP 5y ago
  Yes, as someone in the system, it's all true and an open secret, everyone knows it and we complain about it over some beers all the time. But isn't something like this true everywhere? And blaming the individual is futile. I like Scott Alexander's Moloch [0] concept more useful. If you apply a high standard against your own research regarding thoroughness, not overclaiming etc. never looking at the test set etc., you won't beat the benchmark, which contains entries you know are too good to be true. But if you don't beat them, you'll get rejected. The pace is extreme, lots of groups work on very similar things and it's a race to publish very similar stuff first. As a PhD student you need publications. You are in your late twenties, early thirties, you can't afford to lose several years. The big lie is that there is so much discovery happening as suggested by the thousand upon thousands of papers at big conferences, most of which is never really engaged later on. We have to pretend that all grad students in every group can contributing some actual valuable, bulletproof novel thing to science every few months and pretend that you can have multiple papers at conferences while remaining painfully critical towards your own work. I mean how could it be that people happen to always find something that works out in the end? How can people sit into a project at the start of the PhD and immediately crank out a valuable novel contribution? It isn't realistic, but we have to pretend because this is the background assumption behind a PhD. But the current state of affairs is inflating away the value and prestige of a PhD. It means less and less because everyone knows that it's not just a measure of scientific research skill and knowledge. However, what people are suggesting, namely making it less based on peer review and more on some social media like system is also misguided. It's like the instagrammization of the field: the sexiness, the dopamine hit, the celebrities. But this isn't specific to AI or CS or academia. This is how humans are. It's how status hierarchies are always gamed. Think office politics, actual politics, dating. Atvthe same time actual AI methods are really getting better, it's not a lie and not just hype. It's actually quite paradoxical how well the system works despite the flaws. Vision algorithms work really damn well, its very obvious to anyone who remembers the pre-2010 times. Perhaps there is somewhat of a stagnation in the last few years but also a lot is happening in smaller niches, as the big deep leaning ideas are adapted and applied specifically to particular tasks. The methods work, but the literature is distorted. You need to be an insider to know what to actually believe. [0] https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
• geofft 5y ago
  There's another solution: give up on academia. The model of academia works well when some part of society is willing to fund researchers spending time on open-ended basic research, even if it produces no results. Saying "Don't engage in unethical behavior" is well and good, but if the incentives are towards unethical behavior, you shouldn't expect very much. Academic funding and employment is very closely linked to the number of published papers. The article says that a grad student who publishes three papers a year is a professor's dream - but on reflection, it should be clear that this is not because three papers a year is necessarily good science, it's just a good metric. The professor's lab looks good for publishing so much, the professor can ask for more funding from grant agencies, and the student is likely to get a good faculty job which reflects well on the professor. So it's extraordinarily difficult to stay in the field, have a well-funded lab, hire enough grad students, and get tenure if you're optimizing for the quality of your science over the number of publications. The "publish or perish" culture (literally, in at least one case, it seems) isn't driven by the practitioners; it's driven by the requirements of getting grants and getting academic jobs in an increasingly competitive market. Now in some fields you do need the facilities from a university to do your research, but for CS in general - and especially for AI - you'll be just fine in industry. All you need is a pile of cloud VMs, and industry can get you that. Perhaps you also need interesting data and real-world problems; industry can get you that too. Society today underfunds academia and overfunds industry. You, the individual researcher, are not going to be able to fix this. Go where the incentives are better aligned for you. (And if you're a Ph.D. student who sees no future in your life after being complicit in fraud, please, please reach out to industry. Your prior publications are nowhere near as high-stakes as in an academic job; you can contact the journal, get it retracted, drop out of the program, and have a great life ahead of you. What saddens me is that the student who allegedly took his own life was probably so deeply surrounded by academia that he didn't know there's a world who won't judge him for having one fewer paper on his CV and will even look on him positively for reporting fraud and getting his advisor into career trouble.) --- That said, my argument is undermined a bit because the papers OP denounces have coauthors in industry. So I don't have an answer for why those coauthors went along with it.
  • bonoboTP 5y ago
    Depends on what industry you're talking about. Those that publish are working according to the same practices. Google, Facebook et al. need to pass peer review too and they care about papers as a dick measuring contest.
• goalieca 5y ago
  I ran into these issues in an adjacent field a generation ago. Many candidly spoke to me about having to be a ‘team player’. This is all due to the criteria for survival and graduation. Let’s be honest, not every idea will work and this does not reflect badly on the researcher who came up with it. But the only way to graduate and put food on the table is to keep getting funding through novel papers that have great results.
  • StandardFuture 5y ago
    "Novel papers" with "great results".
  • MattGaiser 5y ago
    > this does not reflect badly on the researcher who came up with it. > But the only way to graduate and put food on the table is to keep getting funding through novel papers that have great results. So in the way in which it matters, it does reflect badly on the researcher who came up with it.
    • bonoboTP 5y ago
      Well, does it reflect badly on a startup founder if their idea doesn't work out? But since things are subtler in science, it's possible to sell stuff and pollute the literature even if it didn't really work out. To do otherwise is altruistic but ultimately outcompeted by those with less qualms about it.
      • MattGaiser 5y ago
        > Well, does it reflect badly on a startup founder if their idea doesn't work out? Maybe not in SV, but I have heard plenty of people tell startup founders to "get a real job" or refer to trying a startup as a "figuring it out phase." When a startup some friends founded failed, they had to deal with people wondering if they chose the startup route as they couldn't get real jobs upon graduation. So, yes, in some places.
    • goalieca 5y ago
• bonoboTP 5y ago
  Either way, even if the problem will be solved it will be the small guys who get screwed. The top dogs have built their careers on this bullshit, now they pull up the ladder and would want to make the playing field harder for anyone else. Typical strategy also with startup companies that skirt the dark gray zone of the law when they are small, but then when they extract enough money, they suddenly demand regulation so the rest of the up and coming are smacked down.
  • Igelau 5y ago
    That was my gut feeling about the ACM article that the author here refers to. The vague exposé in ACM is made by someone in the comfortable position of "Royce Family Professor of Teaching Excellence in Computer Science at Brown University" about people who aren't anywhere near that. It is like saying: please stop treading water so desperately, you are splashing those of us in the lifeboats.
• baybal2 5y ago
  The motivation behind it is very simple: eye-watering salaries in the field. Moneyed people are ready to pay these salaries because a lot of them have succumbed to hype equating "AI" (that phrase alone is complete bullshit) to something bordering on "magic," and bringing equally magical money making opportunities. The prospect for business owners to get rid of white collar workers in a manner not unlike how machines swept blue collar workers is just this much irresistible.
  • StandardFuture 5y ago
    So, in other words, there really is not any "publish or perish" in academia. It's simply that to outcompete your fellow graduate students in getting a top job in industry (or a tenured position in academia) you need to have the most publications with the most "novelty". And that can be justified at any cost. So, is this, at the end of the day, that the ratio of students willing to study graduate level studies, and jobs that require graduate level studies is dramatically imbalanced? Sometimes, we as humans like to avoid the obvious discussion because it is the most uncomfortable discussion to have. The truth is that we need more restrictions and an even higher bar on education that artificially pipes the best of the best in corresponding numbers to estimated availability of jobs in 4-5 years. I am pretty sure that the medical world already does this.
    • bonoboTP 5y ago
      Yes, the would be the outside solution if you were an all seeing God. But in the real world we have to make do with local incentives. And that is leading to a ballooning of academia, professors want more grad students for more papers and more prestige, some of whom also turn into profs and it snowballs. It's great while the music is playing, but then when winter comes back, what will all these people do then? Things are already insanely competitive to the degree that many profs even say they couldn't make it into a graduate program now, you already need top conference papers just to get accepted at grad school (in better groups). In a way, I think a lot of the internal squabbles regarding diversity etc. in the AI community come from the animosity due to a shrinking cake and more hungry mouths.
    • 5y ago
• anonymousDan 5y ago
  The author mentions a collusion ring in AI, but I'm pretty sure the article refers to a situation that arose in the computer architecture community (which itself happened quite a while back).
• tlb 5y ago
  The idea that a field can be reformed by making it worse until it suddenly faces a reckoning and emerges much better is ... I don't know where people keep getting the idea that this might work. It has never worked in any field ever in history. The thing that can happen: fields gradually split into rigorous and non-rigorous camps. Like with evidence-based medicine, or chemistry/alchemy. Depending on the field, either might prevail in the market. Medical research and bridge-building are mostly rigorous, programming is mostly non-rigorous. AI/ML has a range of rigor levels, from fairly good to total crap. I think people in the field have a reasonable idea which is which. It's frustrating to people outside the field that think they can just take the technique with highest reported performance numbers and expect good results.
  • josalhor 5y ago
    Well, I wonder if this could force people to start publishing their code so others can replicate the results. This would help greatly with the kinds of issues that are being discussed: - It would be much harder to hide makeup results and cherry-picked seeds - Useless research would not be as easy to hide - The real-world impact of the research would be more valued. - Researchers would be forced to recheck their code for it to be presentable, which could be a net benefit in terms of finding mistakes - and so on.. I do imagine this could harm the number of replications of the research. However, I think there can be a net positive effect from such a policy. What do you think?
    • the_snooze 5y ago
      I'm active in the security and privacy field, and some conferences have started "artifact reviews" [1][2], where authors can optionally submit the data and code that go along with their work. It's definitely an improvement, but the devil is in the details. Should these be mandatory? If they're mandatory, then what quality standards should artifacts meet? Who's ensuring those standards are followed? How should proprietary code and sensitive human-subjects research data be handled? There's also the question of what code should be made public: Is it just the analysis that produced the metrics on the paper (e.g., R scripts, Jupyter notebooks, etc.)? Or should it also include the data collection and pre-processing? How about the code that didn't produce immediately publishable results, but could be useful for future work? [1] https://petsymposium.org/artifacts.php https://petsymposium.org/artifacts.php [2] https://www.usenix.org/conference/usenixsecurity21/call-for-artifacts https://www.usenix.org/conference/usenixsecurity21/call-for-...
• deleted 5y ago
  [deleted]
• marsven_422 5y ago
  Government funding...what you expect?
• gher-shyu3i 5y ago
  How much of this fraud is committed by Chinese? I've known several people in academia who told me about blatant cheating and fraud committed by them, their culture does not frown upon this behavior, and now everyone is paying the price.
• zitterbewegung 5y ago
  I think we need to rethink the core concept of Universities having publish or perish as a part of being a Professor. The reason why fraud exists is because of the incentive structure and due to the nature of research you have intelligent people that will game the system. So, we have to figure out how to change the incentive structure so that professors don't attempt to perform Academic Fraud or we h ave to figure out if fraud does exist after the fact. I have tried to reproduce ML / Deep learning research. I have a few heuristics that don't really tell me if fraud exists but either they just can't release the data due to special circumstances that I can understand or it isn't kosher. The biggest one is that you have to register or ask them for their test data. Another one is that the paper has been published for more than two years and there are no citations from other work.
  • bonoboTP 5y ago
    Keeping test data private is good practice actually. It keeps from hyperparameter optimization on the test set. Yes theoretically the dataset authors might still do it, but it's still better for the field as a whole to keep some test data secret and only use it for evaluation. Registering before getting data is common, so they can make you agree to some terms. It's not suspicious in itself. There are no hard and fast rules of thumb. You can look at the group, the people involved, the country, the writing style, the quality and thoroughness of ablation, is there code available etc. But nothing is 100%.
  • ModernMech 5y ago
    > So, we have to figure out how to change the incentive structure so that professors don't attempt to perform Academic Fraud or we h ave to figure out if fraud does exist after the fact. One possible way: increase funding to the arts and sciences. The total NSF budget was ~$8B, or 0.03% of US GDP. We could double that and no one would even notice. Increased funding means more money to go around, means higher acceptance rates for grants, means less incentive to write fraudulent papers. The best part is, you have the power to actually change this. Write your congressperson, vote for a congressperson who pledges to do this, run for congress yourself, or organize other people to do this with you.
• lettergram 5y ago
  I manage an applied research group in the machine learning space, particularly AutoML and synthetic data generation. That means is we read the latest literature and create applications. Sometimes we build new research, most of the time we patent something new. But the goal is to get research to production as fast and as impactful as possible. The first thing I do when sorting research (papers, blog posts, presentations, etc) is sort the research by “B.S.”, “maybe B.S.”, “probably real”. If you look at research papers in computer science, half are what have the following problem: “running a simulation over and over until the authors get the results They like”. I can’t tell you how many papers with thousands of citations are just blatantly wrong. They don’t run corrections based on the number of simulations they run, they don’t take into account other variables, etc This isn’t limited to CS either. Biology, economics, environmental sciences, etc all suffer the same fate. The worst part about it, is it’s PEER REVIEWED. Meaning, others agree this is the way to do things, which is why I don’t trust academia almost at all.
  • Q57C3HYc7g 5y ago
    Maybe this is a dumb question, but what do you mean by "run corrections based on the number of simulations they run"?
    • L-four 5y ago
      Say you run 5 simulations and when comparing to the standard approach as a percent the samples are [-1%, -1%, +2%, +2%, +%6]. Your new method is not 6% better the standard it's at best +1.6% better.
    • Jabbles 5y ago
      For example, https://en.wikipedia.org/wiki/Look-elsewhere_effect https://en.wikipedia.org/wiki/Look-elsewhere_effect and more generally https://en.wikipedia.org/wiki/Multiple_comparisons_problem https://en.wikipedia.org/wiki/Multiple_comparisons_problem
  • goalieca 5y ago
    I despised how data sets and code were considered a gold-mine and researchers would often refuse to release them lest they give up a serious paper publishing advantage for their group. More often than not, i found code full of serious errors and completely lacking basic test cases and sanity. Sometimes the data sets were much larger than the subset chosen for publication.
• cryptica 5y ago
  One big problem in our society is that there is a Darwinian selection process occurring which selects for psychopathy. It's only going to get worse. Psycopaths are attracted to power and are willing to do anything for it so they are more likely to get it. Power comes with money. Money allow psychopaths to have more children. The children are more likely to be psychopaths too so they will also end up in positions of power with more money. As automation increases, the system can afford to support an increasing number of psychopaths and they quickly take up all available positions of power via ruthless means. Psychopaths end up occupying all the important political and judicial positions. Altruistic people who follow the rules can't afford to compete in the market place, they have fewer children, they are discriminated against by psychopaths who recognize that they are different from themselves. Altruists become extinct as they are slowly replaced by machines which make altruism a redundant character trait; not necessary for a 'functioning' society. Now that we have machines to automate everything, the economy will no longer rely on 'exploitees' to do the hard value-adding work. Then a whole new class of ultra-psychopaths (e.g. serial killers) will become more common and proceed to wipe out the regular psycopaths through violent means. Humans will get progressively worse and end up driving themselves to extinction. There's not going to be any robot uprising; humans will make themselves extinct before that happens.
  • Igelau 5y ago
    Anecdata and not medical advice, but I used to think up a lot of things like this before I started taking SSRIs.
    • deleted 5y ago
      [deleted]
    • s5300 5y ago
      Well, the unfortunate part is that he's at least somewhat correct, and it's been found that psychopaths are much less likely to seek/adhere to mental health treatment such as SSRIs.
  • selimthegrim 5y ago
    You need to lay off the Peter Watts.
• aabaker99 5y ago
  I'm really happy to see this article here. I quit my PhD where I was working on AI partly because of the "mundane day-to-day fraud" that the author and I observed in the field. Once you've read enough papers, you can easily see a kind of recipe to an academic paper. Present a new method, compare against a baseline, discuss the differences in the results found. To me, some researchers just follow this recipe and do not think about the steps critically. You see figures that plot two categories of points that are purportedly following different distributions but the error bars (or even more of the distributions) overlap. But a p-value says they are significant. So that's good enough to publish! I felt a lot of anxiety about sharing my research in this environment and felt pressure to make my work seem like it was the best solution for every problem when in reality there are more nuances.
  • oldnews193 5y ago
    > I felt a lot of anxiety about sharing my research in this environment and felt pressure to make my work seem like it was the best solution for every problem when in reality there are more nuances. I am curious what kind of pressure you are referring to. It's your work and your decision how to present it to the world. As far as other people's research is concerned, academics can express their opinions by participating in peer review and expressing their opinions vocally at program-committee meetings (for instance, proposing artifact evaluation as a part of the review process).
    • voakbasda 5y ago
      I had the same experience as GP. I dropped out of a PhD program after witnessing widespread low-level fraud. I published a single paper with my advisor. I asked questions that a peer reviewer would have asked, and my concerns were basically ignored. It was more important to publish than be accurate and precise with wording (partly due to length limits imposed by the publication). Pushing back harder likely would have had a deleterious effect on my progress in the program. It would have been career suicide. Worse, the topic was nothing more than rehashed results from a paper he published years earlier. There was really nothing even worth publishing. This was but one example that showed me that academia is endemic with fraud. Eventually I killed that career path, because I could never participate in such a fundamentally corrupt system. There will be no reform here without Revolution.
• weasel_words 5y ago
  The actual "smoking gun" is three levels deep: 1) https://cacm.acm.org/magazines/2021/6/252840-collusion-rings-threaten-the-integrity-of-computer-science-research/fulltext https://cacm.acm.org/magazines/2021/6/252840-collusion-rings... then 2) https://medium.com/@tnvijayk/potential-organized-fraud-in-acm-ieee-computer-architecture-conferences-ccd61169370d https://medium.com/@tnvijayk/potential-organized-fraud-in-ac... then 3) https://huixiangvoice.medium.com/evidence-put-doubts-on-the-ieee-acms-investigation-991a6d50802a https://huixiangvoice.medium.com/evidence-put-doubts-on-the-... Not to try and dissuade this internet mob...but instead of doing the whole internet-mob-get-eyeballs-to-my-blog-by-rehashing-someone-elses-blog thing, get to basics and be precise like a scientist would be. Examples. Forensics. Details.
• turbinerneiter 5y ago
  If this, and the stuff coming up in this thread, is true about academia ... Trump is our fault. How can we tell the people to trust science, if science is a clusterfuck of underpaid PhD students who are incentivized to cheat?
• Cyril_HN 5y ago
  In undergrad, my professor explained to me in very candid terms (from the position of being a generous mentor) that the optimal path to success in the Arts and Social Sciences is: 1. Find a niche only a couple of people operate in. 2. Make friends with them at all costs and work in their area. 3. Review each others work, amassing enormous citations in highly respected (albeit sometimes niche) journals.
  • a9h74j 5y ago
    I saw something similar in an academic's blogged advice ten or fifteen ago: "At graduate level, you should cultivate [peers and a concentration] such that your intellectual correspondence is publishable." I took that as plausible enough at the time to mention the advice once to a graduate student in philosophy. I did not take it at the time as immediately translatable into a [simply translated?] phrase I have heard since: "friendship corruption." There are some video lectures from a writing consultant employed at UChicago. He accepts that concentrations within fields are to some extent self-defined in the academic game -- in only slightly more generous terms. IIRC his strategic advice for those failing to publish is: learn to frame your abstracts as respecting but advancing the discussion. Perhaps in small enough niches ingratiate yourself would need to be mentioned? Or show that you will play by being selectively generous with citations?? Citations. The academic version of SEO and currency more valuable than money? (Recall where page rank came from.) One kind of power behind institution-sanctioned monsters?
    • galimaufry 5y ago
      > "At graduate level, you should cultivate [peers and a concentration] such that your intellectual correspondence is publishable." ... I did not take it at the time as immediately translatable into a [simply translated?] phrase I have heard since: "friendship corruption." This seems misguided, and I certainly hope that this 'friendship corruption' concept never catches on. There are great papers that started as letters and were later completed by the sender, recipient or both. No one should feel ashamed about that, and no one should feel ashamed of developing friendships with their colleagues.
• YeGoblynQueenne 5y ago
  >> Blackmail your reviewers, bribe your ACs! Fudge your results – or fabricate them entirely! How does one blackmail one's reviewers? Asking for a friend.
• dalbasal 5y ago
  The level of academic unhappiness with publishing and its surrounds is actually surprising to me. First, I never hear the counterargument.. that current publishing is OK. Does it exist, or is everyone unhappy while nothing changes? Second, I'm surprised at how much of a single institution academia seems to be. "Publish or Perish," for example, has to be supported by tenure committees, grant makers and such. Are they all the same? Michael Stonebraker^ suggests that tenure committees accept a limited number of papers on a resume. This is to encourage lower volume, higher quality publication. What stops some tenure committees from implementing such changes. I just don't understand what's locking everything into place. ^https://www.youtube.com/watch?v=DJFKl_5JTnA&t=853s https://www.youtube.com/watch?v=DJFKl_5JTnA&t=853s
  • Al-Khwarizmi 5y ago
    Well, this trend is more or less everywhere but it's not uniform, some countries have much more extreme "publish or perish" cultures than others. For example, in Spain practically everything related to the academic career (promotions, salary decisions, tenure, etc.) is measured by pure bean counting. Actual quality doesn't matter at all, it's all about having X papers in Y journal quartile during Z period. Evaluations are conducted by scientists but they might as well be clerks, because there is usually a grading scale on everything saying "each paper in this quartile gives X points" so the weight of the actual opinions of the evaluators is zero or close to zero (even if they read one of those papers and it's total bullshit, the regulations still say that they have to give it X points). As a result, the kind of subtle fraud described in the post is widespread, with most people gaming metrics and trying to publish in the crappiest possible journals that happen to have a high impact factor. In contrast, for example in France things seem to be more relaxed. I don't know the exact details as I'm not from there, but I have many colleagues from there and I do know that they can actually spend years thinking about very difficult problems, and then publishing a really good paper (even if it's in a journal or conference without such good metrics) and their career seems to be more or less fine. As a result, research coming from France in my field tends to generally be really high quality (IMHO) even though in Spain those people's CVs would be considered bad due to not optimizing metrics. The US seems to be somewhere in between, China looks similar to Spain, Northern European countries seem to be somewhere between the US and France in the spectrum, and so on.
• re-al 5y ago
  The lies in science and academia are out of control. Whether its this example, the replication crisis, the funding crisis - the upshot is that we cannot be confident about the 'reality' that is being presented to us.
• underseacables 5y ago
  I graduated in 2014, and was astonished at how much cheating there was in university. From thefts and bribery, to groups sitting in the back of the class whispering in another language, integrity at the university level has plummeted. When students openly cheat and advance, it’s no wonder that behavior becomes permissible among faculty
• quantum_state 5y ago
  A very funny article ... on a serious note, why would one want to cheat oneself by claiming false things in one’s research endeavor?
• xbar 5y ago
  This is deserving a of a careful read and long consideration. Well said.
• williesleg 5y ago
  Global anything.
• huijzer 5y ago
  I blame the incentives. Academia is not about selling facts, it is about selling papers. This is very similar to newspapers; they are about selling newspapers.
• riversam 5y ago
  I have a story of a CS "professor" and fraud. I am a student at University of the People. It's an online-only school which relies on volunteer instructors who are paid a small honorarium for each class. Instructor involvement varies from instructor-to-instructor, but most act mostly as moderators rather than instructors. This is due to the "peer learning" environment of the school. In 2015, the university either hired or hosted (or had some relationship with) an instructor who was fired a month later. He was fired because he'd lied about his credentials. He claimed to have a PhD from either Stanford or MIT, but had none. Prior to volunteering at the school, he'd worked on building up an online profile for himself. Yes, astroturfing. He wrote a few "academic" articles on Second Life. In one, he claims one of the founders of the game as his co-author or a contributor. He "published" this article to a couple of websites and then he posted it on Wikimedia Commons. It has even been cited in real academic works. He wrote a self-published book on Second Life which he submitted to the Library of Congress. He leverages this quite often. He claimed to hold a "world record" on an ACM ICPC challenge. And he managed to work this into the ACM ICPC article on Italian Wikipedia which has since been removed. He used Freebase and Wikidata to create "info boxes" about himself on search engines. He even spent time writing fake articles promoting himself on a websites including Blasting News and IMDb. On the latter he claims to have created a commercial with Julia Roberts. The best part is that while he was at University of the People he was also running a service to help students cheat services like Turnitin. This was something he prompted on LinkedIn, Reddit, and even Wikipedia. Yes, the traces are still there on Wikipedia. To this day, the guy holds a grudge against the university. He posts fake reviews on TrustPilot. He posts comments on the Reddit sub such as posing as a fake recruiter or going on about the university's subsidiary in Israel. He uses a plethora of socks and IPs to edit the university's Wikipedia pages and engage in edit wars. He has also tried to scrub some of his past actions by getting articles deleted. And this doesn't even get into his attacks on other institutions, attempt to run an Italian NGO/political party, and more. My mind continues to be blown as I dig into his activities.
• fighterpilot 5y ago
  I watched a lecture by an MIT database professor who recommends that the top 10 schools only allow up to 3 papers on CVs for junior faculty applications and 10 for tenure applications. He thinks this will be enough to start a culture shift towards quality over quantity, which could go some way towards addressing the fraud and collusion ring issues by removing the incentive for these behaviors.
  • DistressedDrone 5y ago
    If the universities wanted the problem fixed it would be fixed. I truly don't believe research universities are focused on producing value (other than to themselves).
    • fighterpilot 5y ago
      Apparently it wasn't always like this, and it used to be common for junior faculty hires to have only 0 or 1 publications coming out of their PhD program. Why did the culture deteriorate so much and become so myopically focused on weakly informative metrics like publication count?
      • magila 5y ago
        Because government grants demand it. Private patrons can trust their own judgment when deciding who to fund, but when it's taxpayer money being handed out people are understandably going to demand objective metrics to guard against corruption. In academia the objective metrics of choice are publication and citation count, so here we are.
        • christophilus 5y ago
          This is the answer, or a very large part of it. Incentives are set up such that this is an inevitable outcome.
      • _dps 5y ago
        Two likely contributors: 1) due to population effects, academic positions are much more competitive now than they were in say 1970; if you figure that the top 50 research universities are not generally expanding the number of professors, and that new professors generally also come from those top 50 research universities, then on average a top-50-research-university professor will generate one new such professor in a career, despite having 10-100x as many graduate students (this was different in the 70s when the university system was rapidly expanding). 2) the increasing desire for fairness in hiring and promotion (by itself, a good thing) means that you need to be able to resolve hiring and promotion disputes with something both objective and external to the university (in the same way some undergraduate institutions put more admissions weight on external and objective metrics like standardized tests compared to more easily game-able internal metrics like high school class grades)
• plaidfuji 5y ago
  > Undermining the credibility of computer science research is the best possible outcome for the field, since the institution in its current form does not deserve the credibility that it has. This is obviously somewhat tongue in cheek, but I’m not sure the author appreciates the full implications of how this would play out. Research funding for AI exists in competition with funding for all other fields. Revealing deep-seated BS would serve mostly to decrease funding commitments to AI projects in favor of other fields, or away from R&D entirely. It would be bad for all researchers (but if the true level of fraud is that high, and I have no doubt that it is, this may simply be an inevitable market correction). Other comments in this thread seem to miss the point as well: “why don’t universities select for quality not quantity”? Well, the change has to start with the funding sources for the unis to care.
  • Aperocky 5y ago
    I think this is exactly the implication. The amount of people pouring into AI, or more specifically to Neural Network version of AI is unnatural, and it resembles a bubble in the classic economic sense.
• ModernMech 5y ago
  There's a lot of discussion here about changing the incentive structure of academia to prevent this kind of fraud, but based on the discussion I've read so far I don't think a lot of people here understand the exact incentives that are involved, so I'd like to try to explain how academics actually operate. We've all head of "publish or perish", and there is a general understanding that you have to publish papers to get tenure, so that's where the incentive is. But it's a little more complicated than that. When you are hired as a tenure-track assistant professor at a university, you're given what's called a "startup" package. Assistant professors are essentially startup founders where their company is a lab. Your job as an assistant professor is to use that money to purchase equipment and to recruit PhD students (aka employees) so that you can launch your research agenda. This money pays for: - rent taken off the top (essentially paying for your lab space and other costs the University incurs) - a portion of your own salary - your students' tuition - your students' stipends - conference travel and fees - lab equipment (computers, desks, chairs, machinery, scientific apparatuses). To note here, for every dollar you spend on equipment, the University takes a percentage of that. So if I buy a chair for $100, the University will charge for example $10. This is a sort of tax for being affiliated with the University and getting free use of their resources (internet, libraries, subscriptions, etc.). - As well as other things. Crucially, this money is not free. The expectation is that you will in the next 6 or so years use that money to jumpstart a successful lab operation. A "successful" lab is one that is able to procure significant funding from government and institutional sources. Your first goal is to procure enough funding to recoup the initial startup package the University laid out. That's your bare minimum. Ideally, you want to procure funding far in excess of this by the time you're up for tenure, to prove to the University that your lab will bring in more dollars. Because you've got to earn your keep. Notice there hasn't been much discussion of papers so far. The initial startup package can be quite significant -- on the order of millions of dollars depending on the research agenda. As an assistant professor, you need to bring in millions of dollars from funding agencies like the NSF to make that back in multiples. But the NSF budget was only $8 billion dollars in 2020, and they need to spread that money around to new assistant professors across the entire country. Therefore, funding rates usually range from 10-20% for a proposal, which is quite low when your entire future is depending on it. This is where "publish or perish" comes in. When you are up for tenure, yes they'll be looking at the quantity and quality of your publications. However, you could have 1000 high quality publications, but if you don't bring in significant grant money, you will not be getting tenure. The way you get that grant money is by writing a good research grant proposal, and the way you back up and bolster those proposals is through publications. This is why there is so much fraud in publications. Not getting tenure at this stage means your research career is essentially over. You are as good as fired from your current role, and few Universities will take you on as a tenure track professor when you've already failed at your current institution. At this point you're in your mid to late 30s, and your best years (in terms of research ideas) are already behind you. It's time to enter industry or become a lecturer. So how to fix this? 1. More research funding for government agencies. Higher funding rates means less incentive to commit fraud. 2. A feedback loop of profits from industry to universities. Right now when technological advancements happen at universities, those flow out to the general public, then to companies, who figure out how to monetize them. The profits generated by that monetization are captured entirely by private corporations, even though the foundational innovations on which they are based are funded by Universities and by extension the public. If some of this profit were diverted back into Universities (maybe through increased corporate taxes), then this would help. 3. Lessen the incentive for Universities to essentially become hedge funds with schools attached. My University is currently sitting on a billion dollars that is locked up in investments. Harvard's endowment is currently 40 billion dollars. They could fund the entire country's research agenda 5x over with that. Yet professors are paying their own salaries with public money? And the University gets a cut? This is insane. 4. Offer an alternative off-ramp for academics who don't get tenure. If not getting tenure won't end your career you've spent your entire life building, then the incentive to commit fraud would be lessened as well. Anyway, tldr; there's not enough money to go around, so fraud exists.
• zx2391 5y ago
  I experienced parts of CS academia and it was off-putting to say the least. Full of mediocre people who understood "the system" - not that much different from a direct-marketing operation, or a mafia - all while pestering CS professionals working in the weeds with their crazy-talk. Creating papers out of things, my colleagues and I talk while having a coffee. I'm not joking, intelligent guys get high on their own supply and will sell you ideas, that never gonna fly in the real-world with a straight face - and they will try it over and over again. It's embarassing. Personally, I'm trying out the route of getting the resources myself to do my research, much more work - but at least sane work with a level of independence that I feel is necessary to see things through.
• tester756 5y ago
  I'd add to it: "Statistics done wrong"