3 ms·
Needs more work. >> Right now, the YesNoError website contains many false positives, says Nick Brown, a researcher in scientific integrity at Linnaeus Universi
by YeGoblynQueenne 2y ago
Needs more work.
>> Right now, the YesNoError website contains many false positives, says Nick Brown, a researcher in scientific integrity at Linnaeus University. Among 40 papers flagged as having issues, he found 14 false positives (for example, the model stating that a figure referred to in the text did not appear in the paper, when it did). “The vast majority of the problems they’re finding appear to be writing issues,” and a lot of the detections are wrong, he says.
>> Brown is wary that the effort will create a flood for the scientific community to clear up, as well fuss about minor errors such as typos, many of which should be spotted during peer review (both projects largely look at papers in preprint repositories). Unless the technology drastically improves, “this is going to generate huge amounts of work for no obvious benefit”, says Brown. “It strikes me as extraordinarily naive.”
- dadadad100 2y agoMuch like scanning tools looking for CVEs. There are thousands of devs right this moment chasing alleged vulns. It is early days for all of these tools. Giving papers a look over is an unqualified good as it is for code. I like the approach of keeping it private until the researcher can respond.
- Lerc 2y agoThere are two different projects being discussed here. One Open source effort and one "AI Entrepreneur" effort. YesNoError is the latter project. AI, like Cryptocurrencies faces a lot of criticism because of the snake oil and varying levels of poor applications ranging from the fanciful to outright fraud. It bothers me a bit how much of that critique spreads onto the field as well. The origin of the phrase "snake oil" comes from a touted medical treatment, a field that has charlatans deceiving people to this day. In years past I would have thought it a given that people would not consider a wholesale rejection of healthcare as a field because of the presence of fraud. Post pandemic, with the abundance of conspiracies, I have some doubts. I guess the point I'm making is judge each thing on their individual merits. It might not all be bathwater.
- LiamPowell 2y ago> for example, the model stating that a figure referred to in the text did not appear in the paper, when it did This shouldn't even be possible for most journals where cross-references with links are required as LaTeX or similar will emit an error.
- jraph 2y agoI've never seen this. Usually you don't have the LaTeX source of a paper you cite, you wouldn't know which label to use for the reference, when the cited paper is written in LaTeX at all. Or something changed quite a bit in recent years. Can you link to another paper's Figure 2.2 now, and have LaTeX error out if the link is broken? How does that work?
- LiamPowell 2y agoI assume they're referring to internal references. It does not look like they feed cited papers in to their tool.
- jraph 2y agoOh, ok. I had badly misunderstood the quote.