5 ms·
> This is incredibly good for science. I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for t
by dataflow 5mo ago
> This is incredibly good for science.
I disagree. It's just one darn hallucinated citation for heaven's sake, not fraud or something. It doesn't account for the substance or quality of their work at all. A one-year ban seems plenty sufficient for a minor first time mistake like this. People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress their life or contribute to humanity just because an AI hallucinated a reference one time in their life. That's punitive instead of rehabilitative.
- ajkjk 5mo agoIt's not the kind of mistake that is possible unless you're engaging in fraud anyway.
- dataflow 5mo ago> It's not the kind of mistake that is possible unless you're engaging in fraud anyway. Seriously? You can't fathom an honest researcher asking for AI to find a citation they know exists, and the AI inserting or modifying a citation incorrectly without them realizing? If you find evidence of fraud by all means lay down the hammer. Using a single hallucinated citation like it's some kind of ironclad proxy just because you think they must be committing fraud is insane.
- mixtureoftakes 5mo agoif you're not checking citations in the paper youre publishing AND trusting a non SOTA, hallucination prone ai model to come up with sources for it, its probably for the best of everyone that this paper isn't published. yes there will be rare exceptions but in general i feel like this is a really good addition.
- flexagoon 5mo ago> non SOTA, hallucination prone ai model What SOTA models are not hallucination prone?
- asdff 5mo agoWhy would you ask the ai to find a citation you know exists? Just reach for that citation.
- andrepd 5mo agoIf you are citing a work you paste a citation to that work. If you are bullshitting you ask an AI to come up with a citation. Jesus, there is zero reason to ever "generate a citation" if you are not, in fact, commiting fraud.
- galaxyLogic 5mo agoI much agree. But I wonder shouldn't the citations all be hyperlinks and thus easy to verify?
- no-name-here 5mo agoHow specific are the citations? If it's “Sentence 4 on page 97 supports” or “Paper says ‘___’” then I imagine it would be fairly easy. If it's “(__ page long) paper supports x”, then very difficult?
- fc417fc802 5mo agoThat's like saying that there's zero reason to ever ask an LLM to do basic math for you. Sure you probably shouldn't do that but sometimes it's convenient and so people will inevitably do exactly that regardless of the somewhat frequent wrong answers that are guaranteed to ensue.
- skydhash 5mo agoFraud are always about the convenience of the perpetrator.
- BonoboIO 5mo agoI would not necessarily go as far as calling it fraud, but if you cannot even verify that the reference you are citing actually exists, you are not ready to publish research papers. Deadlines are not an excuse here. Checking whether a cited book, paper, or passage exists is the absolute minimum standard for scientific work, not an optional extra. I have written academic papers myself, and I find it astonishing that people are trying to justify this as if it were some understandable workflow mistake. At that point it is simply slop with academic formatting. A one-year ban is not a lifetime ban. Maybe six months would also have been enough, but the point is that the author gets time to think about whether they should verify references next time. They can also use that time to manually check every other citation.
- BonoboIO 5mo agoVerifying that the reference you cite actually exists is the absolute minimum standard for academic work. It is not optional, not something to skip because of a deadline, and not something to outsource blindly to hallucination-prone AI. If someone cannot meet that bar, they have no business publishing research papers. I have written academic papers myself, and I find it astonishing that people are trying to justify this as if it were some understandable workflow mistake. At that point it is simply slop with academic formatting. Post it on a blog or somewhere else, but do not put it into the scientific record. A one-year ban is not a lifetime ban. Maybe six months would also have been enough, but the author can use that time to think about whether they should verify references next time — and to manually check every other citation.
- dataflow 5mo ago> I have written academic papers myself So have I. That doesn't place me on some pedestal. > A one-year ban is not a lifetime ban. Nobody said it was. Nobody was even objecting to it. Please (re-?)read past the first 8 words of [1], the first 3 sentences of [2], or the first 3 tweets of the original tweet thread. You appear to have imagined (shall we call this a hallucination?) and responded to something that was never claimed. Which is especially unfortunate, given the very topic of the lecture here. [1] https://news.ycombinator.com/item?id=48141324 https://news.ycombinator.com/item?id=48141324 [2] https://news.ycombinator.com/item?id=48142322 https://news.ycombinator.com/item?id=48142322
- ajkjk 5mo agoYes, having AI write something and not checking it yourself is sure to lead to hallucination, hence, it is a fraudulent way to write.
- protocolture 5mo ago>Seriously? You can't fathom an honest researcher asking for AI to find a citation they know exists Assumptions: 1. The entire document is loaded into an AI editor 2. The researcher is asking an AI editor to work on his references 3. The researcher has not checked his own references. This could be avoided at 1, 2 or 3. But even just 1 implies that the researcher knows that they have a hot potato and might critically fuck up and lose all credibility. Being in that scenario and committing to 2 and 3 is at least extreme negligence.
- toofy 5mo agoif an llm does the work, you did not write it or research it, the llm did. you have no business crediting yourself as an author. if someone writes a paper and an entirely different person takes credit for it without even bothering to check if the actual writer just made shit up, they deserve a lifetime ban. seems like a year is a very light punishment.
- brazzy 5mo ago> Seriously? You can't fathom an honest researcher asking for AI to find a citation they know exists, and the AI inserting or modifying a citation incorrectly without them realizing? Indeed I cannot. If you do that, you are not, in fact, an honest researcher. You're a lazy hack.
- wrs 5mo agoA "mistake" would be a typo in a real citation. A hallucinated citation is evidence of just plain laziness and negligence, which taints the entire submission.
- dataflow 5mo agoNo it is not. Seriously. All you need for this to happen is for your lab partner to ask AI to add a missing citation that they are already familiar with at the last minute before a midnight submission deadline, and for the AI to hallucinate something else, and for them to honestly miss this. It does not even imply any involvement on your part, let alone that either of you were lazy or negligent on the actual research or substance of the paper. The lack of any sympathy or imagination here is astounding.
- asdff 5mo agoThere are no deadlines for journal submissions. Even if you felt you were running close to your revisions being due, an email to an editor will probably fix this for you. And what you described is still negligent, not verifying the garbage output bot did not in fact output garbage.
- AnimalMuppet 5mo agoEven more, there are no deadlines for arXiv submissions.
- cozzyd 5mo agoKinda. There is a deadline to appear in the next days posting and this can be important if e.g. you want to get something on the arxiv before a talk or a proposal deadline.
- bigstrat2003 5mo agoThe lack of understanding that you are responsible for the content you create, no matter what tools you use, is what's astounding.
- Loughla 5mo agoDon't use AI? Problem solved?
- patcon 5mo agoA citation is where you derived knowledge... If you haven't checked it and you are submitting something that should represent a ton of labour (and which will consume labour to review), you don't understand what you're doing. It is not just crossing T's and dotting I'd. Your being set behind is less important than the fact that your publishing is setting everyone else behind. Such a banned person is being helped to "step out of the way", and someone more competent will assuredly step forward to consume the limited maintenance labour more thoughtfully
- dataflow 5mo ago> Your being set behind is less important than the fact that your publishing is setting everyone else behind One hallucinated citation does not in any way imply anyone is being left behind. All it means nobody is checked that particular line of the manuscript after it was written. The rest of the paper could still be solid and treated accordingly. If you find evidence of the contrary, of course treat it accordingly, but this is so obviously not that.
- some_furry 5mo agoNo. It's fraud.
- andrepd 5mo agoYou clearly misunderstand. You cite a work in your paper because you have read that work, and build upon it or want to refer to it to back up a specific claim. Generating references is fraud period, because you are implying that you have read a work when in fact you just asked an AI "please insert some reference-shaped text here" to make it look like a proper paper. It is sadly not a necessary, but certainly a VERY sufficient, reason to conclude a paper is fraudulent.
- dumpsterdiver 5mo ago> One hallucinated citation does not in any way imply anyone is being left behind. The parent said “setting” others behind, which refers to lost time. Being “left” behind implies a degraded trajectory, which is defined not by time lost, but by the final destination. Different but related things (e.g. lost time can indeed affect your final destination, for instance, after growing old correcting a scourge of hallucinated citations - which should have been table stakes all along).
- conartist6 5mo agoYes, it is fraud
- toast0 5mo ago> It's just one darn hallucinated citation for heaven's sake, not fraud or something. It is fraud. > It doesn't account for the substance or quality of their work at all. References are part of the work. If you're making up the references, what else are you making up? > People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress their life or contribute to humanity just because an AI hallucinated a reference one time in their life. A one year ban is not permanent. Having a negative consequence for making poor decisions seems like an inducement to learn from the mistake? In an ideal world, one would be keeping notes on references used while doing the research that lead to writing the paper. Choosing not to do that is one poor decision. Having a positive outlook, if asking an AI to provide references that may have been missed, one should at least verify the references exist and are relevant. Choosing not to do that is also a poor decision, even if one did take notes on references used while researching.
- blazespin 5mo agoit's very silly, but not a big deal. Arxiv is becoming irrelevant these days anyways. In fact would be better if they just banned AI, so we could just get off the luddite platforms. Automated research is the future, end of story. And really it couldn't have come out at a better time, given the increasingly diminishing returns on human powered research.
- andrepd 5mo agoPoe's law striking hard.
- AnimalMuppet 5mo agoIf automated research is the future, it has to be research, not making stuff up. Which of those two does "hallucinated references" fit into?
- dataflow 5mo ago> It is fraud. No, it is emphatically not. Fraud requires intent to deceive. > A one year ban is not permanent. ...what text are you reading? Nobody was calling the one-year ban permanent, or even against it. I was literally in favor of it in my comment. I explicitly said it is already plenty sufficient. What I said is there's no need to go beyond that. My entire gripe was that they very much are going beyond that with a permanent penalty. Did you completely miss where they said "...followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue"?
- mianos 5mo agoYou are being ironic right?
- goolz 5mo agoIf you cannot be bothered to check your references when writing academic quality papers then you have no place writing them in the first place. The punishment is not chopping off a finger, it is a polite reminder to do the bare minimum.
- slashdave 5mo agoWell, in the good old days, when we have refereed journals, it would be part of the publishing process.
- themafia 5mo ago> There's no need to permanently cripple someone's ability to progress their life or contribute to humanity I don't think you need to publish on arXive to contribute meaningfully to humanity. > That's punitive instead of rehabilitative. Unfortunately science is competitive. Yours is a race to the bottom where the people who can afford the most expensive models and who are least concerned with the truth can publish the most papers and benefit financially and professionally by doing so. This is not a zero sum arena, grant money and opportunities will possibly be rewarded to them, and not to another team who is producing more careful and genuine output.
- vhantz 5mo agoWhat's the difference between a "hallucinated" citation and consciously inserting reference to a non-existent paper and hopping it goes unnoticed? How do we determine which one was done consciously and which was "a minor first time mistake"? Your standards are lower than what they would accept at my high-school. Seriously. And generally, if you are generating papers with LLMs, let other LLMs read them. Why would we waste human hours considering something that was generated? At this point publish your prompt because that's the actual work you're doing.
- simianparrot 5mo agoIn science, one hallucinated reference can corrupt the entire rest of the work. So you're completely wrong.
- gizajob 5mo agoAnd every piece of work in future which cites the paper with the hallucinated reference.
- nkrisc 5mo agoIt’s easy to avoid this whole issue: write the paper yourself.
- redsocksfan45 5mo ago[dead]
- avanwyk 5mo agoWe got pretty far without a single hallucinated citation (because they simply didn't exist). It's really simple: check your references, or don't bother of publishing. It's not a high hurdle.