2 ms·
"The researchers hypothesize that GPT-style generative models, and similar types of language generation frameworks, have been used to produce much of the text i
by gwern 5y ago
"The researchers hypothesize that GPT-style generative models, and similar types of language generation frameworks, have been used to produce much of the text in the flagged papers; however, the way that a generative model abstracts its sources makes this difficult to prove, and the chief evidence lies in a common-sense evaluation of poor and unnecessary synonyms, and a meticulous examination of the submission’s logical coherence."
This sounds extremely unlikely.
First, scientific fraudsters are typically extremely lazy; they do not use fancy methods like writing generative models of fake data to ensure the numbers, say, add up to 100%, much less neural nets. (Do we believe that someone would go to the effort of setting up GPT to generate fake papers and then be satisfied with results like 'profound neural organization'? Easier to write such garbage by hand!)
Second, the given examples are so bad I'm insulted on behalf of GPT: 'profound neural organization' looks like a word-by-word calque using a thesaurus or particularly antiquated machine translation (in a "the vodka is good, but the meat is rotten" way), no contemporary AI involved; GPT would not generate such garbage.
Third, if GPT was generating these papers, it would be easy to detect by looking for top-k signatures, using GPT-targeted classifiers (they get >90% accuracy), or looking for stylistic tics of GPT; that they don't provide any specific GPT-like aspects of the text suggests there are none.
- jka 5y ago"Do we believe that someone would go to the effort of setting up GPT to generate fake papers..." I'd say it depends on what their desired outcome is. If they know that other people are using AI to assess the literature about a particular topic, then the content only needs to be good enough to pass that analytical AI's sniff test and bias the corpus in the way that the owner wants it to. If the owner is lazy, perhaps that affords some time against this kind of pollution. But the costs and level of effort involved will continue to reduce.