3 ms·

Highlighting research that's useful is probably too difficult, but highlighting research that definitely isn't should be well within the bounds of the best exis

Highlighting research that's useful is probably too difficult, but highlighting research that definitely isn't should be well within the bounds of the best existing reasoning LLMs. There are a lot of common patterns in papers that you learn to look for after a while, and it's now absolutely within reach of automation. For example, papers whose conclusions section doesn't match the conclusions in the abstract are a common problem, likewise papers that contain fake citations (the cited document doesn't support the claim, or has nothing to do with the claim, or sometimes is even retracted). For that matter you can get a long way with pre-neural NLP. This database tracks suspicious papers using hand-crafted heuristics: https://dbrech.irit.fr/pls/apex/f?p=9999:1 https://dbrech.irit.fr/pls/apex/f?p=9999:1:::::: What you'll quickly realize though is that none of this matters. Detection happens every day, the problem is that the people who run grant-funded science itself just don't care. At all. Not even a little bit. You can literally send them 100% bulletproof evidence of scientific fraud and they'll just ignore it unless it goes viral and the sort of media they like to read begins asking questions. If it merely goes viral on social media, or if it gets picked up by FOX News or whatever, then they'll not only ignore it but double down and defend or even reward the perpetrators.

LarsDu88 2y ago
I think you misunderstand my proposal. I am not describing a fraud classifier. I am describing fine-tuning an embedding space based on papers with known fraud. The content of the fraudelent paper which includes information such as authorship and other citations can be made to exist on an embedding space. Supervised fine-tuning on labels will alter the shape of that embedding space. The resulting embeddings that would come from such a fine-tuned model would generate different clusterings of papers than what you'd get than if you did not have the labeled fraudelent data at all.

LarsDu88 2y ago

I think you misunderstand my proposal. I am not describing a fraud classifier. I am describing fine-tuning an embedding space based on papers with known fraud. The content of the fraudelent paper which includes information such as authorship and other citations can be made to exist on an embedding space. Supervised fine-tuning on labels will alter the shape of that embedding space. The resulting embeddings that would come from such a fine-tuned model would generate different clusterings of papers than what you'd get than if you did not have the labeled fraudelent data at all.