4 ms·

> It is fraud. I think we are talking semantics here. While fraud does require intention to deceive, I get the sentiment that hallucinated citations shouldn't

by ksd482 5mo ago

> It is fraud. I think we are talking semantics here. While fraud does require intention to deceive, I get the sentiment that hallucinated citations shouldn't be dismissed as simply carelessness. It should be something stronger than that: gross negligence or something MUCH stronger! There should absolutely be repercussions for this. But let's not call it fraud. That word is reserved for something specific. EDIT: someone else said "reckless disregard" equals intent or something to that effect. So I looked it up. It appears so that is the case. "Reckless Disregard Equals Intent" in legal language. But I am not sure if this particular clause should apply here. Perhaps it depends on what kind of research is being published? For e.g., if it is related to medical science and has a real consequence on people's health, we can then apply this?

gpm 5mo ago
The intent to deceive is there. The deception is lying when you submit it that it is a scholarly piece of work in which amongst many other things you know the citations are accurate. This false representation was knowingly and intentionally made at the time of submission. The citation being incorrect is merely the proof of deception not the (relevant) deception itself. Fraud is the correct description provided (and this is practically a guarantee) you intended to benefit from the submission of the paper (e.g. by bolstering your resume).
- fc417fc802 5mo ago
  If I violate the letter of the ToS when clicking submit you can correctly argue that I have technically committed fraud! Yet that is almost never what anyone actually means when having discussions like this one. Fraud in a scientific context generally refers to fabricated research results. At least personally I agree with GP that hallucinated citations are generally something akin to laziness thus not fraud but rather some sort of professional negligence.
  - gpm 5mo ago
    Fraud in the scientific world has generally taken the form of fabricated results, but I don't agree that the word has transitioned away from the common and legal meaning of deception in order to get a benefit. Even if it had though, I'd be perfectly comfortable calling this fraud in this discussion based on the common meaning of the word. Just because we're talking about a scientific context does not mean we need to use the scientific-jargon versions of words - we're not in a scientific context ourselves. --- And I'd disagree that this is just about the "letter of the ToS". While that is perhaps a necessary component in order to prove the deception, this is really about the cultural expectations of the community that merely happened to have been encoded in the ToS. The fraud would still occur without the ToS, it would merely be next to impossible to show you didn't simply misunderstand the cultural norms and what your actions would lead others to believe.
    - fc417fc802 5mo ago
      I disagree with your implicit assertion regarding the common meaning of the term in this context. I believe that the term fraud as commonly used when discussing things in a scientific context has always (for at least my entire life) been taken to refer to knowingly and intentionally falsified research results (also falsified appointments, falsified affiliations, falsified authorships, etc). > deception in order to get a benefit. The point being that reckless or negligent conduct is not commonly taken to constitute deception. There's a reason we have different terms for these things. Sure, you can say "well he exhibited reckless disregard for his professional duties when he opted not to bother reading the citation section that the LLM shat out, and reckless disregard is sufficient to meet the legal bar for fraud, and also the ToS specifically says that you certify that you validated all references manually so bam! two counts of fraud legally speaking" and you wouldn't be wrong but the distinction between "legally fraud" and "fraud as is commonly meant when talking about scientific papers" is essential to effective conversation in this particular instance. > Just because we're talking about a scientific context does not mean we need to use the scientific-jargon versions of words The context is essential because (obviously) it affects how people interpret the meaning of your words. A fraudulent submission to a scientific journal has a specific and well understood meaning in common usage. If you still disagree with me imagine polling a bunch of tenured career researchers about what they would think if they read the statement "X caught submitting a fraudulent paper to journal Y". I can just about guarantee you that none of them are imagining hallucinated citations.
fc417fc802 5mo ago
I think (though might well be misunderstanding) that reckless disregard is taken to be an intentional choice but that it does not imply that the outcome itself was intentional. The difference between intentionally doing something that you know for a fact has a high risk of failure but you can't necessarily predict the outcome versus intentionally seeking a particular legally disallowed outcome. But what LPisGood was saying is that reckless disregard (as opposed to explicit intent) is sufficient to meet the legal bar for fraud.
eqvinox 5mo ago
I do believe this policy is appropriate to deal with the reckless disregard of posting hallucinated references. It's a conscious decision to not take the time to check your AI output, and instead waste a whole bunch of other people's time letting them essentially do that for you in duplicate. Feels like that should disqualify you from participation for a bit. Intent or no intent.
- dataflow 5mo ago
  > Feels like that should disqualify you from participation for a bit. Intent or no intent. Exactly! For a bit! Yet this is not for a bit! This is a lifetime disqualification, and that's been my entire grip the whole time! Is nobody reading this? "The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue."
  - lelanthran 5mo ago
    That doesn't sound like a lifetime ban to me.
    - dataflow 5mo ago
      What? Then how long are they disqualifying them from submitting prior to acceptance, if not lifetime? It certainly doesn't say 1 year or something.
      - lelanthran 5mo ago
        > Then how long are they disqualifying them from submitting prior to acceptance, if not lifetime? Well, "lifetime ban" means "you are not allowed back in". Their ban specifically allows you back in (after a specified period) subject to fulfilling a single constraint. It's conditional acceptance back in, which is not the same as a lifetime ban which is unconditional.
        5mo ago

by ksd482 5mo ago

gpm 5mo ago

The intent to deceive is there. The deception is lying when you submit it that it is a scholarly piece of work in which amongst many other things you know the citations are accurate. This false representation was knowingly and intentionally made at the time of submission. The citation being incorrect is merely the proof of deception not the (relevant) deception itself. Fraud is the correct description provided (and this is practically a guarantee) you intended to benefit from the submission of the paper (e.g. by bolstering your resume).

fc417fc802 5mo ago
If I violate the letter of the ToS when clicking submit you can correctly argue that I have technically committed fraud! Yet that is almost never what anyone actually means when having discussions like this one. Fraud in a scientific context generally refers to fabricated research results. At least personally I agree with GP that hallucinated citations are generally something akin to laziness thus not fraud but rather some sort of professional negligence.
- gpm 5mo ago
  Fraud in the scientific world has generally taken the form of fabricated results, but I don't agree that the word has transitioned away from the common and legal meaning of deception in order to get a benefit. Even if it had though, I'd be perfectly comfortable calling this fraud in this discussion based on the common meaning of the word. Just because we're talking about a scientific context does not mean we need to use the scientific-jargon versions of words - we're not in a scientific context ourselves. --- And I'd disagree that this is just about the "letter of the ToS". While that is perhaps a necessary component in order to prove the deception, this is really about the cultural expectations of the community that merely happened to have been encoded in the ToS. The fraud would still occur without the ToS, it would merely be next to impossible to show you didn't simply misunderstand the cultural norms and what your actions would lead others to believe.
  - fc417fc802 5mo ago
    I disagree with your implicit assertion regarding the common meaning of the term in this context. I believe that the term fraud as commonly used when discussing things in a scientific context has always (for at least my entire life) been taken to refer to knowingly and intentionally falsified research results (also falsified appointments, falsified affiliations, falsified authorships, etc). > deception in order to get a benefit. The point being that reckless or negligent conduct is not commonly taken to constitute deception. There's a reason we have different terms for these things. Sure, you can say "well he exhibited reckless disregard for his professional duties when he opted not to bother reading the citation section that the LLM shat out, and reckless disregard is sufficient to meet the legal bar for fraud, and also the ToS specifically says that you certify that you validated all references manually so bam! two counts of fraud legally speaking" and you wouldn't be wrong but the distinction between "legally fraud" and "fraud as is commonly meant when talking about scientific papers" is essential to effective conversation in this particular instance. > Just because we're talking about a scientific context does not mean we need to use the scientific-jargon versions of words The context is essential because (obviously) it affects how people interpret the meaning of your words. A fraudulent submission to a scientific journal has a specific and well understood meaning in common usage. If you still disagree with me imagine polling a bunch of tenured career researchers about what they would think if they read the statement "X caught submitting a fraudulent paper to journal Y". I can just about guarantee you that none of them are imagining hallucinated citations.

fc417fc802 5mo ago

I think (though might well be misunderstanding) that reckless disregard is taken to be an intentional choice but that it does not imply that the outcome itself was intentional. The difference between intentionally doing something that you know for a fact has a high risk of failure but you can't necessarily predict the outcome versus intentionally seeking a particular legally disallowed outcome. But what LPisGood was saying is that reckless disregard (as opposed to explicit intent) is sufficient to meet the legal bar for fraud.

eqvinox 5mo ago

I do believe this policy is appropriate to deal with the reckless disregard of posting hallucinated references. It's a conscious decision to not take the time to check your AI output, and instead waste a whole bunch of other people's time letting them essentially do that for you in duplicate. Feels like that should disqualify you from participation for a bit. Intent or no intent.

dataflow 5mo ago
> Feels like that should disqualify you from participation for a bit. Intent or no intent. Exactly! For a bit! Yet this is not for a bit! This is a lifetime disqualification, and that's been my entire grip the whole time! Is nobody reading this? "The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue."
- lelanthran 5mo ago
  That doesn't sound like a lifetime ban to me.
  - dataflow 5mo ago
    What? Then how long are they disqualifying them from submitting prior to acceptance, if not lifetime? It certainly doesn't say 1 year or something.
    - lelanthran 5mo ago
      > Then how long are they disqualifying them from submitting prior to acceptance, if not lifetime? Well, "lifetime ban" means "you are not allowed back in". Their ban specifically allows you back in (after a specified period) subject to fulfilling a single constraint. It's conditional acceptance back in, which is not the same as a lifetime ban which is unconditional.
      - 5mo ago