5 ms·

Four quick thoughts: 1. Bad grammar in a paper is incredibly irritating, which is particularly unfair for people who are non-native English speakers. If ChatGP

Four quick thoughts: 1. Bad grammar in a paper is incredibly irritating, which is particularly unfair for people who are non-native English speakers. If ChatGPT can help people to convert awkward English into nicer work without changing the ideas or confabulating, that seems like a win. I don't see why people need to "declare" their usage of ChatGPT for these purposes. Is that a standard now? 2. The major issue here is that the authors copied a response and accidentally also brought some button text with it. This is given to imply that people aren't reading ChatGPT's output. But I think this is just as likely to be benign. People do all sorts of silly nonsense in papers (including me), like leaving author comments in final drafts. It's bad and sloppy, but it isn't usually a crime. 3. Many papers on public servers are junky. Many journals and conferences are equally junky. Search for any bad string and you'll find it everywhere, and non-scientists will be scandalized. 4. I like the people at Retraction Watch and think they're doing the Lord's work. However sometimes I worry that immersing yourself in wickedness makes you see wickedness everywhere, and also creates an (unconscious) incentive to sensationalize bad stuff. I hope they're aware of this.

stayfrosty420 3y ago
how is (2) likely to be benign and not plagiarism? It is completely different from leaving an author comment in a draft...such a weird take.
- matthewdgreen 3y ago
  If I type a series of bullet points into ChatGPT and ask it to re-phrase them in better English prose, am I plagiarizing? If it's not plagiarism but I screw up at copy/pasting, does it become plagiarism? I don't know! It is not something I personally would ever do because I like my own writing. But I'm also not going to throw that accusation at ESL authors who are using a writing tool (like Autocorrect-on-steroids) to convey original ideas to an audience, at least not until we've had a much longer community discussion on using these tools. ETA I would urge you not to throw the death penalty at people whose crime may be more akin to shoplifting. This is bad both for the shoplifters and for people who want to fight serious crime. Plagiarism is one of the most serious accusations in science: casually tossing it around like this risks our ability to enforce it against serious offenders.
  - thfuran 3y ago
    >like Autocorrect-on-steroids It's a difference in quality large enough to constitute a difference in kind. Autocorrect nominally takes text in your own words as typed and converts them into your words as intended. Chatgpt used as you describe does not produce text in the author's own words.
    - matthewdgreen 3y ago
      As children in grade school we are taught a series of rules about plagiarism that are designed to make us extremely sensitive about what we copy from. We're taught this because if you let a group of children do whatever they want, they'll copy stuff straight off Wikipedia or each other and thus they won't learn how to write properly. So we teach them that writing every word yourself is the most important goal to aspire to. As an adult in a scientific field, what actually matters to me is having original scientific ideas conveyed to me in an efficient manner. I'm not really interested in applying arbitrary elementary-school rules on authors, as long as they aren't stealing ideas and text from other people (thus misallocating credit in ways that are harmful to progress.) I think as a community we should proceed from these principles, then work the details out from there.
      - dragonwriter 3y ago
        > As an adult in a scientific field, what actually matters to me is having original scientific ideas conveyed to me in an efficient manner. I'm not really interested in applying arbitrary elementary-school rules on authors, as long as they aren't stealing ideas and text from other people. The elementary school rules, at least the ones I weas taught decades ago, are pretty much “don’t steal ideas and text from other people”. Like, literally, other than for formatting, there were exactly three rules avout plagiarism I was taught on this in elementary school, and only one of them becomes slightly fuzzier outside of that context: (1) if you use exactly wording from someone else, put it in quotes and cite the source, (2) if you use an idea from someone else, cite the source but don't ise quotes, and (3) if an idea comes from three or more independent sources, you don’t need to cite it, it can be treated as “common knowledge”. Outside of elementary school, yeah, whether something is genuinely common knowledge or should be cited to multiple sources is fuzzier and context dependent, but the other two rules apply pretty directly.
deleted 3y ago
[deleted]
not2b 3y ago
The authors could be required to declare only to the reviewers that ChatGPT was used for the purpose of cleaning up the language. Reviewers could then be alert to check for artifacts like this, verify that no references were hallucinated, etc. Perhaps a footnote would be added to mention that ChatGPT was used in this way.
JohnFen 3y ago
I don't know. The use of an LLM to help produce something isn't necessarily a bad thing -- but in my mind, it does reduce my trust in what the thing is saying. It means that if I actually care about what the paper (or whatever) is talking about, I need to spend a lot more time confirming what it's saying. If I don't care much about what the paper is talking about, I'm more likely just to discount the paper and move on.
StableAlkyne 3y ago
The challenge here is if their English skills are so poor that they need a ML model to help make it readable, their English skills are likely so poor that they will not spot grammatical changes that adjust the meaning of the work. That's a problem when the next person finds that paper on Google Scholar and cites it - you get a telephone game in the literature. It's hard to remove bad "facts" from the literature after a few people cite it, since you also need to go after a whole chain of papers, hoping their journals are willing to issue a correction. God help you if it's in the intro section and the authors complain because of academia's "corrections are bad" mindset. Anything that increases the BS rate of the literature is a bad thing. Also, if you go to contact the author or view a talk, and suddenly their English is broken and completely different than what they wrote, you start questioning whether they wrote it or paid someone to without disclosing it. IMO it's worth disclosing for many of the same reasons you disclose funding, ethics approval, author contributions, and conflicts of interest at the end of an article: it's metadata that protects your credibility as an author by showing that you're not doing anything underhanded.
breakwaterlabs 3y ago
The article also gives the example of the phrase "as an AI language model, I..." being found in published papers. The fact that this slipped by the paper's author, their editor, *and the journal review* is not benign. I agree with the author here: is no one reading these papers before putting their names behind them? Is the entire journal system entirely fraudulent?
- deleted 3y ago
  [deleted]
- StableAlkyne 3y ago
  Peer review is a mixed bag. It's the best system we've come up with, but still can be hit or miss. Depends on how much the journal is willing to push the authors (predatory journals don't give a heck, prestigious journals usually care more because they've got a rep to uphold). Reviewers can be anyone from an expert in the field, a competitor who gives you a hard time (usually they're not actively malicious though), to some random prof doing it because the editor couldn't find anyone else, or even a grad student in the lab of a prof (it's a common training exercise). I've had articles with anywhere from 1-3 reviewers, there is no standard beyond "the editor is a basic crackpottery filter, there is an external reviewer, and a copy editor makes it fit for viewing"
  - breakwaterlabs 3y ago
    If the system is not catching "I am a language model", I have zero confidence in its ability to detect crackpottery, much less more insidious things like P-hacking.
- deleted 3y ago
  [deleted]

stayfrosty420 3y ago

how is (2) likely to be benign and not plagiarism? It is completely different from leaving an author comment in a draft...such a weird take.

matthewdgreen 3y ago
If I type a series of bullet points into ChatGPT and ask it to re-phrase them in better English prose, am I plagiarizing? If it's not plagiarism but I screw up at copy/pasting, does it become plagiarism? I don't know! It is not something I personally would ever do because I like my own writing. But I'm also not going to throw that accusation at ESL authors who are using a writing tool (like Autocorrect-on-steroids) to convey original ideas to an audience, at least not until we've had a much longer community discussion on using these tools. ETA I would urge you not to throw the death penalty at people whose crime may be more akin to shoplifting. This is bad both for the shoplifters and for people who want to fight serious crime. Plagiarism is one of the most serious accusations in science: casually tossing it around like this risks our ability to enforce it against serious offenders.
- thfuran 3y ago
  >like Autocorrect-on-steroids It's a difference in quality large enough to constitute a difference in kind. Autocorrect nominally takes text in your own words as typed and converts them into your words as intended. Chatgpt used as you describe does not produce text in the author's own words.
  - matthewdgreen 3y ago
    As children in grade school we are taught a series of rules about plagiarism that are designed to make us extremely sensitive about what we copy from. We're taught this because if you let a group of children do whatever they want, they'll copy stuff straight off Wikipedia or each other and thus they won't learn how to write properly. So we teach them that writing every word yourself is the most important goal to aspire to. As an adult in a scientific field, what actually matters to me is having original scientific ideas conveyed to me in an efficient manner. I'm not really interested in applying arbitrary elementary-school rules on authors, as long as they aren't stealing ideas and text from other people (thus misallocating credit in ways that are harmful to progress.) I think as a community we should proceed from these principles, then work the details out from there.
    - dragonwriter 3y ago
      > As an adult in a scientific field, what actually matters to me is having original scientific ideas conveyed to me in an efficient manner. I'm not really interested in applying arbitrary elementary-school rules on authors, as long as they aren't stealing ideas and text from other people. The elementary school rules, at least the ones I weas taught decades ago, are pretty much “don’t steal ideas and text from other people”. Like, literally, other than for formatting, there were exactly three rules avout plagiarism I was taught on this in elementary school, and only one of them becomes slightly fuzzier outside of that context: (1) if you use exactly wording from someone else, put it in quotes and cite the source, (2) if you use an idea from someone else, cite the source but don't ise quotes, and (3) if an idea comes from three or more independent sources, you don’t need to cite it, it can be treated as “common knowledge”. Outside of elementary school, yeah, whether something is genuinely common knowledge or should be cited to multiple sources is fuzzier and context dependent, but the other two rules apply pretty directly.

deleted 3y ago

[deleted]

not2b 3y ago

The authors could be required to declare only to the reviewers that ChatGPT was used for the purpose of cleaning up the language. Reviewers could then be alert to check for artifacts like this, verify that no references were hallucinated, etc. Perhaps a footnote would be added to mention that ChatGPT was used in this way.

JohnFen 3y ago

I don't know. The use of an LLM to help produce something isn't necessarily a bad thing -- but in my mind, it does reduce my trust in what the thing is saying. It means that if I actually care about what the paper (or whatever) is talking about, I need to spend a lot more time confirming what it's saying. If I don't care much about what the paper is talking about, I'm more likely just to discount the paper and move on.

StableAlkyne 3y ago

The challenge here is if their English skills are so poor that they need a ML model to help make it readable, their English skills are likely so poor that they will not spot grammatical changes that adjust the meaning of the work. That's a problem when the next person finds that paper on Google Scholar and cites it - you get a telephone game in the literature. It's hard to remove bad "facts" from the literature after a few people cite it, since you also need to go after a whole chain of papers, hoping their journals are willing to issue a correction. God help you if it's in the intro section and the authors complain because of academia's "corrections are bad" mindset. Anything that increases the BS rate of the literature is a bad thing. Also, if you go to contact the author or view a talk, and suddenly their English is broken and completely different than what they wrote, you start questioning whether they wrote it or paid someone to without disclosing it. IMO it's worth disclosing for many of the same reasons you disclose funding, ethics approval, author contributions, and conflicts of interest at the end of an article: it's metadata that protects your credibility as an author by showing that you're not doing anything underhanded.

breakwaterlabs 3y ago

The article also gives the example of the phrase "as an AI language model, I..." being found in published papers. The fact that this slipped by the paper's author, their editor, *and the journal review* is not benign. I agree with the author here: is no one reading these papers before putting their names behind them? Is the entire journal system entirely fraudulent?

deleted 3y ago
[deleted]
StableAlkyne 3y ago
Peer review is a mixed bag. It's the best system we've come up with, but still can be hit or miss. Depends on how much the journal is willing to push the authors (predatory journals don't give a heck, prestigious journals usually care more because they've got a rep to uphold). Reviewers can be anyone from an expert in the field, a competitor who gives you a hard time (usually they're not actively malicious though), to some random prof doing it because the editor couldn't find anyone else, or even a grad student in the lab of a prof (it's a common training exercise). I've had articles with anywhere from 1-3 reviewers, there is no standard beyond "the editor is a basic crackpottery filter, there is an external reviewer, and a copy editor makes it fit for viewing"
- breakwaterlabs 3y ago
  If the system is not catching "I am a language model", I have zero confidence in its ability to detect crackpottery, much less more insidious things like P-hacking.
deleted 3y ago
[deleted]