4 ms·

I just witnessed LLM (ab)use coming from one graduate student (not the first to do it and definitely not the last), where they submitted a conference paper draf

I just witnessed LLM (ab)use coming from one graduate student (not the first to do it and definitely not the last), where they submitted a conference paper draft for their coauthors and advisors to review short before a deadline, with completely regurgitated material plus hallucinations backed up by multiple non-existent citations. The problem is every coauthor wants to increase submissions, LLMs are great at making something that looks OK at first glance, and people have low(er) expectations for a conference paper. A recipe for disaster. Extrapolate a bit and there are LLM written papers being peer reviewed by LLMs, but fear it not, even if they are accepted the will not be cited because LLMs are hallucinating citations that better support their arguments! And then there is the poor researcher, just a beginner, writing a draft, simple but honest material getting lost in all this line noise, or worse out, feeding it.

mccoyb 1y ago
Anecdotally: I tried hook and crook to get the best flagship model at the time (Opus) to help with technical writing for a submission. First, these models are not good at technical writing at all. They have no sense of the weight of a single sentence, they just love to blather. Second, they can't keep the core technical story consistent throughout their completions. In other words, they can't "keep the main thing the main thing". I had an early draft with AI writing, but by the time we submitted our work -- there was not a single piece of AI writing in the paper. And not without trying, I really did some iterations on trying to carefully craft context, give them a sense of the world model in which they needed to evaluate their additions, yada yada. For clear and concise technical communication, it's a waste of time right now.
- jszymborski 1y ago
  I'm so happy I have pre-LLM publications and blog posts to prove that my blathering isn't because I'm lazy and used Claude, it's just how I write (i.e., badly).
  - sroussey 1y ago
    poorly…
    - jszymborski 1y ago
      It would be v. funny if I got that wrong, but I do feel the need to point out that "badly" is indeed grammatically correct here because this is HN and pedantry is always on topic. People over-correct and feel like they can't use "badly" because there is "feeling badly" discourse [0], but that pertains to "feeling" being a linking verb. "Write" is just your bog standard verb for which "badly", an adverb, is a totally valid modifier. [0] https://www.merriam-webster.com/grammar/do-you-feel-bad-or-feel-badly https://www.merriam-webster.com/grammar/do-you-feel-bad-or-f...
      - dcminter 1y ago
        This is just a by the by, but in British English "feeling poorly" mostly means that you are ill. Amusingly it's become slightly euphemistic, so if someone is "a bit poorly" they probably have sniffles or a minor fever. If they are "very poorly" then you probably heard it from a hospital and they're just about dead. Thus "I feel badly" ... "ok, what did you do?" vs. "I feel poorly" ... "ok, I'll get a bucket."
pyuser583 1y ago
"there are LLM written papers being peer reviewed by LLMs, but fear it not, even if they are accepted the will not be cited because LLMs are hallucinating citations that better support their arguments!" I just shared that with several friends, and we are all having a good laugh. Thank you very much.

mccoyb 1y ago

Anecdotally: I tried hook and crook to get the best flagship model at the time (Opus) to help with technical writing for a submission. First, these models are not good at technical writing at all. They have no sense of the weight of a single sentence, they just love to blather. Second, they can't keep the core technical story consistent throughout their completions. In other words, they can't "keep the main thing the main thing". I had an early draft with AI writing, but by the time we submitted our work -- there was not a single piece of AI writing in the paper. And not without trying, I really did some iterations on trying to carefully craft context, give them a sense of the world model in which they needed to evaluate their additions, yada yada. For clear and concise technical communication, it's a waste of time right now.

jszymborski 1y ago
I'm so happy I have pre-LLM publications and blog posts to prove that my blathering isn't because I'm lazy and used Claude, it's just how I write (i.e., badly).
- sroussey 1y ago
  poorly…
  - jszymborski 1y ago
    It would be v. funny if I got that wrong, but I do feel the need to point out that "badly" is indeed grammatically correct here because this is HN and pedantry is always on topic. People over-correct and feel like they can't use "badly" because there is "feeling badly" discourse [0], but that pertains to "feeling" being a linking verb. "Write" is just your bog standard verb for which "badly", an adverb, is a totally valid modifier. [0] https://www.merriam-webster.com/grammar/do-you-feel-bad-or-feel-badly https://www.merriam-webster.com/grammar/do-you-feel-bad-or-f...
    - dcminter 1y ago
      This is just a by the by, but in British English "feeling poorly" mostly means that you are ill. Amusingly it's become slightly euphemistic, so if someone is "a bit poorly" they probably have sniffles or a minor fever. If they are "very poorly" then you probably heard it from a hospital and they're just about dead. Thus "I feel badly" ... "ok, what did you do?" vs. "I feel poorly" ... "ok, I'll get a bucket."

pyuser583 1y ago

"there are LLM written papers being peer reviewed by LLMs, but fear it not, even if they are accepted the will not be cited because LLMs are hallucinating citations that better support their arguments!" I just shared that with several friends, and we are all having a good laugh. Thank you very much.