4 ms·

> But the human reviewers didn't do much better: they correctly identified only 68% of the generated abstracts and 86% of the genuine abstracts. They incorrectl

> But the human reviewers didn't do much better: they correctly identified only 68% of the generated abstracts and 86% of the genuine abstracts. They incorrectly identified 32% of the generated abstracts as being real and 14% of the genuine abstracts as being generated. Isn't the headline actually "Abstracts written by ChatGPT fool scientists 1/3 of the time"? Having never written one myself, wouldn't the abstract be the place where ChatGPT shines, being able to write unsubstantiated information confidently? I imagine getting into the meat of the paper would quickly reveal issues.

lelandfe 4y ago
> I imagine getting into the meat of the paper would quickly reveal issues This is a tautology: the thing that can be validated can be validated.
- SketchySeaBeast 4y ago
  I think it has an important difference for ChatGPT - it likes to generate numbers that make absolutely no sense. A human that lies will try to generate sufficiently correct data to convince. ChatGPT often won't even make an attempt to produce values that fit.
  - daveguy 4y ago
    I guess that's good for us meat beings. Better for an AI to incompetently lie than competently lie. I wonder if having AI models available would make it significantly easier to identify material created by that model. Seems it would be easier, but it would still be a big problem. Nets can be trained to identify ai vs not ai on a given model. And that's without needing access to the model weights, just training examples. But when there are N potential models... Edit: Second paragraph added later.
  - neaden 4y ago
    ChatGPT right now also likes to make up fake citations when you ask it how it knows something, which could be checked quickly.
    - InCityDreams 4y ago
      ...request an example [I've tried, avail is none].
poulsbohemian 4y ago
>they correctly identified only 68% of the generated abstracts and 86% of the genuine abstracts. I think you and I are basically in alignment... what this tells me is that 14% of real abstracts are so bad that other human beings call their BS. Meanwhile, this AI stuff is kinda working 32% of the time in generating legitimately interesting ideas. So at that point - yeah, that sounds about right. The 32% is still so low that it shows AI is not anywhere near maturity, whereas 14% of human-generated is crap. And, yeah - a short blurb like an abstract seems to be exactly the kind of text that ChatGPT is conditioned to do well generating. As others below note - once a human starts reading the rest, the alarm bells trigger.
- sdenton4 4y ago
  (Buuuut let us also enjoy the fact that ML can generate a decent-looking abstract to a scientific paper 32% of the time. This represents massive progress; I would venture that most humans cannot write such an abstract convincingly.)
  - tduberne 4y ago
    I would argue the opposite. Give anyone who can write reasonnably the abstract of a few dozen papers in a given field, and I am confident they would be able to produce a convincing bullshit abstract. Most abstracts in a given field sound extremely similar, and are mostly keyword dropping with an ounce of self-promotion. I actually think that the only reason ChatGPT did not produce a higher rate of convincing abstracts is that the reaserchers assessing them knew they had a high probability to look at an AI result and were thus extra careful. Most of the abstracts correctly labelled as AI would probably be considered legit (which does not mean good) if sent to a conference without warning.
- seanmcdirmid 4y ago
  It really would depend on the quality of the work/paper. I bet mis-identification by humans of hand written or generated abstracts goes down in higher tier conferences vs. lower tier ones, since the writing standards in lower tier conferences can be pretty bad (well, in CS at least).
cyanydeez 4y ago
What should result is a more fortified peer review system.
maegul 4y ago
All true I think. Interesting thing is how many researchers read just the abstract of papers. I’d say it’s quite common to read just the abstract and some future captions, and only dig into details on a few papers.
- Al-Khwarizmi 4y ago
  That's the main purpose of the abstract. There are several levels of reading a paper: checking the title, reading the abstract, reading intro/conclusions and skimming over the table, or reading the whole thing. You start at the first level and continue or stop depending on the interest of the paper for your work (and available time). This works very well to more or less stay on top of things with today's frantic publication pace in many disciplines.

lelandfe 4y ago

> I imagine getting into the meat of the paper would quickly reveal issues This is a tautology: the thing that can be validated can be validated.

SketchySeaBeast 4y ago
I think it has an important difference for ChatGPT - it likes to generate numbers that make absolutely no sense. A human that lies will try to generate sufficiently correct data to convince. ChatGPT often won't even make an attempt to produce values that fit.
- daveguy 4y ago
  I guess that's good for us meat beings. Better for an AI to incompetently lie than competently lie. I wonder if having AI models available would make it significantly easier to identify material created by that model. Seems it would be easier, but it would still be a big problem. Nets can be trained to identify ai vs not ai on a given model. And that's without needing access to the model weights, just training examples. But when there are N potential models... Edit: Second paragraph added later.
- neaden 4y ago
  ChatGPT right now also likes to make up fake citations when you ask it how it knows something, which could be checked quickly.
  - InCityDreams 4y ago
    ...request an example [I've tried, avail is none].

poulsbohemian 4y ago

>they correctly identified only 68% of the generated abstracts and 86% of the genuine abstracts. I think you and I are basically in alignment... what this tells me is that 14% of real abstracts are so bad that other human beings call their BS. Meanwhile, this AI stuff is kinda working 32% of the time in generating legitimately interesting ideas. So at that point - yeah, that sounds about right. The 32% is still so low that it shows AI is not anywhere near maturity, whereas 14% of human-generated is crap. And, yeah - a short blurb like an abstract seems to be exactly the kind of text that ChatGPT is conditioned to do well generating. As others below note - once a human starts reading the rest, the alarm bells trigger.

sdenton4 4y ago
(Buuuut let us also enjoy the fact that ML can generate a decent-looking abstract to a scientific paper 32% of the time. This represents massive progress; I would venture that most humans cannot write such an abstract convincingly.)
- tduberne 4y ago
  I would argue the opposite. Give anyone who can write reasonnably the abstract of a few dozen papers in a given field, and I am confident they would be able to produce a convincing bullshit abstract. Most abstracts in a given field sound extremely similar, and are mostly keyword dropping with an ounce of self-promotion. I actually think that the only reason ChatGPT did not produce a higher rate of convincing abstracts is that the reaserchers assessing them knew they had a high probability to look at an AI result and were thus extra careful. Most of the abstracts correctly labelled as AI would probably be considered legit (which does not mean good) if sent to a conference without warning.
seanmcdirmid 4y ago
It really would depend on the quality of the work/paper. I bet mis-identification by humans of hand written or generated abstracts goes down in higher tier conferences vs. lower tier ones, since the writing standards in lower tier conferences can be pretty bad (well, in CS at least).

cyanydeez 4y ago

What should result is a more fortified peer review system.

maegul 4y ago

All true I think. Interesting thing is how many researchers read just the abstract of papers. I’d say it’s quite common to read just the abstract and some future captions, and only dig into details on a few papers.

Al-Khwarizmi 4y ago
That's the main purpose of the abstract. There are several levels of reading a paper: checking the title, reading the abstract, reading intro/conclusions and skimming over the table, or reading the whole thing. You start at the first level and continue or stop depending on the interest of the paper for your work (and available time). This works very well to more or less stay on top of things with today's frantic publication pace in many disciplines.