6 ms·

> It's just one darn hallucinated citation for heaven's sake, not fraud or something. It is fraud. > It doesn't account for the substance or quality of their

by toast0 6mo ago

> It's just one darn hallucinated citation for heaven's sake, not fraud or something. It is fraud. > It doesn't account for the substance or quality of their work at all. References are part of the work. If you're making up the references, what else are you making up? > People make mistakes and a good fraction of them can learn from those mistakes. There's no need to permanently cripple someone's ability to progress their life or contribute to humanity just because an AI hallucinated a reference one time in their life. A one year ban is not permanent. Having a negative consequence for making poor decisions seems like an inducement to learn from the mistake? In an ideal world, one would be keeping notes on references used while doing the research that lead to writing the paper. Choosing not to do that is one poor decision. Having a positive outlook, if asking an AI to provide references that may have been missed, one should at least verify the references exist and are relevant. Choosing not to do that is also a poor decision, even if one did take notes on references used while researching.

blazespin 6mo ago
it's very silly, but not a big deal. Arxiv is becoming irrelevant these days anyways. In fact would be better if they just banned AI, so we could just get off the luddite platforms. Automated research is the future, end of story. And really it couldn't have come out at a better time, given the increasingly diminishing returns on human powered research.
- andrepd 6mo ago
  Poe's law striking hard.
- AnimalMuppet 6mo ago
  If automated research is the future, it has to be research, not making stuff up. Which of those two does "hallucinated references" fit into?
dataflow 6mo ago
> It is fraud. No, it is emphatically not. Fraud requires intent to deceive. > A one year ban is not permanent. ...what text are you reading? Nobody was calling the one-year ban permanent, or even against it. I was literally in favor of it in my comment. I explicitly said it is already plenty sufficient. What I said is there's no need to go beyond that. My entire gripe was that they very much are going beyond that with a permanent penalty. Did you completely miss where they said "...followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue"?
- zeusdclxvi 6mo ago
  If you are using AI-hallucinated references in scientific papers then there is some obvious intent to deceive there
- NiloCK 6mo ago
  > No, it is emphatically not. D Fraud requires intent to deceive. I'm about as pro AI-as-a-research--and-writing-assistant and anti AI-witchhunt as they come, but I simply cannot parse what I've quoted here. Posting slop to arxiv is blatant deception. Posting an article is an attestation that the article is a genuine engagement with the literature. If you're posting things to arxiv that are not sincere engagements with the literature, you are attempting to deceive.
  - protocolture 6mo ago
    >I'm about as pro AI-as-a-research--and-writing-assistant and anti AI-witchhunt as they come, but I simply cannot parse what I've quoted here. Ditto. And its only 1 year. Like its about the most reasonable thing they could have done.
    - dataflow 6mo ago
      > And its only 1 year No, it emphatically is not just a year! It's perpetual, and that's literally been my entire point this whole time. If it was just one year I would've had no complaints - and I made that clear from the very first comment! What part of "...followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue..." is everyone here reading and still somehow interpreting to be limited to 1 year?
ksd482 6mo ago
> It is fraud. I think we are talking semantics here. While fraud does require intention to deceive, I get the sentiment that hallucinated citations shouldn't be dismissed as simply carelessness. It should be something stronger than that: gross negligence or something MUCH stronger! There should absolutely be repercussions for this. But let's not call it fraud. That word is reserved for something specific. EDIT: someone else said "reckless disregard" equals intent or something to that effect. So I looked it up. It appears so that is the case. "Reckless Disregard Equals Intent" in legal language. But I am not sure if this particular clause should apply here. Perhaps it depends on what kind of research is being published? For e.g., if it is related to medical science and has a real consequence on people's health, we can then apply this?
- gpm 6mo ago
  The intent to deceive is there. The deception is lying when you submit it that it is a scholarly piece of work in which amongst many other things you know the citations are accurate. This false representation was knowingly and intentionally made at the time of submission. The citation being incorrect is merely the proof of deception not the (relevant) deception itself. Fraud is the correct description provided (and this is practically a guarantee) you intended to benefit from the submission of the paper (e.g. by bolstering your resume).
  - fc417fc802 6mo ago
    If I violate the letter of the ToS when clicking submit you can correctly argue that I have technically committed fraud! Yet that is almost never what anyone actually means when having discussions like this one. Fraud in a scientific context generally refers to fabricated research results. At least personally I agree with GP that hallucinated citations are generally something akin to laziness thus not fraud but rather some sort of professional negligence.
    - gpm 6mo ago
      Fraud in the scientific world has generally taken the form of fabricated results, but I don't agree that the word has transitioned away from the common and legal meaning of deception in order to get a benefit. Even if it had though, I'd be perfectly comfortable calling this fraud in this discussion based on the common meaning of the word. Just because we're talking about a scientific context does not mean we need to use the scientific-jargon versions of words - we're not in a scientific context ourselves. --- And I'd disagree that this is just about the "letter of the ToS". While that is perhaps a necessary component in order to prove the deception, this is really about the cultural expectations of the community that merely happened to have been encoded in the ToS. The fraud would still occur without the ToS, it would merely be next to impossible to show you didn't simply misunderstand the cultural norms and what your actions would lead others to believe.
godelski 6mo ago
> In an ideal world, one would be keeping notes on references used In a far less than ideal world authors are referencing papers they've at least read the title and abstract of. In an ideal world, authors would be only referencing works they have read in their entirety. I don't think we need to live in the ideal world[0], but let's also not pretend the ideal world is even remotely out of reach. Let's also be honest that in the current setting a lot of citations are being used to encourage a work be accepted more than they are being used because of their utility to the paper. The average ML paper now is 8 pages and has >50 citations. That's crazy [0] References can be entire textbooks, which is potentially too high of a bar
- withinboredom 6mo ago
  Even as a human, you can still fuck up references. I submitted a paper with a reference author as Elisio because I couldn’t read my own handwriting. After submitting, I double checked all the references through an LLM. It pointed out that their name was actually Enrique. Yes, you should probably double check your references before submitting, not after. Point is, I didn’t even trust the LLM at first. But after verifying the mistake, I was embarrassed af. I resubmitted with the fixes before it went live, but ultimately, what’s the difference between “mistake” and “hallucination”?
  - lhoff 6mo ago
    I assume they won’t ban anyone automatically without a way to object. Using your example, i wouldn’t assume they would enforce the ban if you object and explain your typo and if the corrected citation actually says what you cited. Mistakes like these are explainable a completely hallucinated citation is usually not.
  - emil-lp 6mo ago
    Sounds like you could use a tool like Zotero. With proper bibliography management tools, everything (that has one) is centered around the DOI. In fact, if a DOI is present, it's trivial to verify authors, title, venue, year, pages etc. Of course, some older and more obscure papers won't have a DOI, but the vast majority of research work has.
rossjudson 6mo ago
If you write your own paper (mostly) and choose your own references (because you've actually read the papers) you won't have a problem.
jruohonen 6mo ago
> In an ideal world, one would be keeping notes on references used while doing the research that lead to writing the paper. Choosing not to do that is one poor decision. In this book https://news.ycombinator.com/item?id=44022957 https://news.ycombinator.com/item?id=44022957 there is this passage on p. 127: "Any author citing another paper should be required to provide proof that they a) possess a copy of that paper, b) have read that paper, c) have read the paper carefully."

by toast0 6mo ago

blazespin 6mo ago

it's very silly, but not a big deal. Arxiv is becoming irrelevant these days anyways. In fact would be better if they just banned AI, so we could just get off the luddite platforms. Automated research is the future, end of story. And really it couldn't have come out at a better time, given the increasingly diminishing returns on human powered research.

andrepd 6mo ago
Poe's law striking hard.
AnimalMuppet 6mo ago
If automated research is the future, it has to be research, not making stuff up. Which of those two does "hallucinated references" fit into?

dataflow 6mo ago

> It is fraud. No, it is emphatically not. Fraud requires intent to deceive. > A one year ban is not permanent. ...what text are you reading? Nobody was calling the one-year ban permanent, or even against it. I was literally in favor of it in my comment. I explicitly said it is already plenty sufficient. What I said is there's no need to go beyond that. My entire gripe was that they very much are going beyond that with a permanent penalty. Did you completely miss where they said "...followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue"?

zeusdclxvi 6mo ago
If you are using AI-hallucinated references in scientific papers then there is some obvious intent to deceive there
NiloCK 6mo ago
> No, it is emphatically not. D Fraud requires intent to deceive. I'm about as pro AI-as-a-research--and-writing-assistant and anti AI-witchhunt as they come, but I simply cannot parse what I've quoted here. Posting slop to arxiv is blatant deception. Posting an article is an attestation that the article is a genuine engagement with the literature. If you're posting things to arxiv that are not sincere engagements with the literature, you are attempting to deceive.
- protocolture 6mo ago
  >I'm about as pro AI-as-a-research--and-writing-assistant and anti AI-witchhunt as they come, but I simply cannot parse what I've quoted here. Ditto. And its only 1 year. Like its about the most reasonable thing they could have done.
  - dataflow 6mo ago
    > And its only 1 year No, it emphatically is not just a year! It's perpetual, and that's literally been my entire point this whole time. If it was just one year I would've had no complaints - and I made that clear from the very first comment! What part of "...followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue..." is everyone here reading and still somehow interpreting to be limited to 1 year?

ksd482 6mo ago

> It is fraud. I think we are talking semantics here. While fraud does require intention to deceive, I get the sentiment that hallucinated citations shouldn't be dismissed as simply carelessness. It should be something stronger than that: gross negligence or something MUCH stronger! There should absolutely be repercussions for this. But let's not call it fraud. That word is reserved for something specific. EDIT: someone else said "reckless disregard" equals intent or something to that effect. So I looked it up. It appears so that is the case. "Reckless Disregard Equals Intent" in legal language. But I am not sure if this particular clause should apply here. Perhaps it depends on what kind of research is being published? For e.g., if it is related to medical science and has a real consequence on people's health, we can then apply this?

gpm 6mo ago
The intent to deceive is there. The deception is lying when you submit it that it is a scholarly piece of work in which amongst many other things you know the citations are accurate. This false representation was knowingly and intentionally made at the time of submission. The citation being incorrect is merely the proof of deception not the (relevant) deception itself. Fraud is the correct description provided (and this is practically a guarantee) you intended to benefit from the submission of the paper (e.g. by bolstering your resume).
- fc417fc802 6mo ago
  If I violate the letter of the ToS when clicking submit you can correctly argue that I have technically committed fraud! Yet that is almost never what anyone actually means when having discussions like this one. Fraud in a scientific context generally refers to fabricated research results. At least personally I agree with GP that hallucinated citations are generally something akin to laziness thus not fraud but rather some sort of professional negligence.
  - gpm 6mo ago
    Fraud in the scientific world has generally taken the form of fabricated results, but I don't agree that the word has transitioned away from the common and legal meaning of deception in order to get a benefit. Even if it had though, I'd be perfectly comfortable calling this fraud in this discussion based on the common meaning of the word. Just because we're talking about a scientific context does not mean we need to use the scientific-jargon versions of words - we're not in a scientific context ourselves. --- And I'd disagree that this is just about the "letter of the ToS". While that is perhaps a necessary component in order to prove the deception, this is really about the cultural expectations of the community that merely happened to have been encoded in the ToS. The fraud would still occur without the ToS, it would merely be next to impossible to show you didn't simply misunderstand the cultural norms and what your actions would lead others to believe.

godelski 6mo ago

> In an ideal world, one would be keeping notes on references used In a far less than ideal world authors are referencing papers they've at least read the title and abstract of. In an ideal world, authors would be only referencing works they have read in their entirety. I don't think we need to live in the ideal world[0], but let's also not pretend the ideal world is even remotely out of reach. Let's also be honest that in the current setting a lot of citations are being used to encourage a work be accepted more than they are being used because of their utility to the paper. The average ML paper now is 8 pages and has >50 citations. That's crazy [0] References can be entire textbooks, which is potentially too high of a bar

withinboredom 6mo ago
Even as a human, you can still fuck up references. I submitted a paper with a reference author as Elisio because I couldn’t read my own handwriting. After submitting, I double checked all the references through an LLM. It pointed out that their name was actually Enrique. Yes, you should probably double check your references before submitting, not after. Point is, I didn’t even trust the LLM at first. But after verifying the mistake, I was embarrassed af. I resubmitted with the fixes before it went live, but ultimately, what’s the difference between “mistake” and “hallucination”?
- lhoff 6mo ago
  I assume they won’t ban anyone automatically without a way to object. Using your example, i wouldn’t assume they would enforce the ban if you object and explain your typo and if the corrected citation actually says what you cited. Mistakes like these are explainable a completely hallucinated citation is usually not.
- emil-lp 6mo ago
  Sounds like you could use a tool like Zotero. With proper bibliography management tools, everything (that has one) is centered around the DOI. In fact, if a DOI is present, it's trivial to verify authors, title, venue, year, pages etc. Of course, some older and more obscure papers won't have a DOI, but the vast majority of research work has.

rossjudson 6mo ago

If you write your own paper (mostly) and choose your own references (because you've actually read the papers) you won't have a problem.

jruohonen 6mo ago

> In an ideal world, one would be keeping notes on references used while doing the research that lead to writing the paper. Choosing not to do that is one poor decision. In this book https://news.ycombinator.com/item?id=44022957 https://news.ycombinator.com/item?id=44022957 there is this passage on p. 127: "Any author citing another paper should be required to provide proof that they a) possess a copy of that paper, b) have read that paper, c) have read the paper carefully."