7 ms·

For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general

For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity. It isn't long before you realise that the adage that this is just autocomplete on steroids is true. However programming a computer to do this isn't easy. In a previous job I had dealing with plagiarism detectors and soon realised how garbage they were (and also how easily fooled they are - but that is another story). The staff soon realised what garbage these tools are so if a student accused of plagiarism decided to argue back then the accusation would be quietly dropped.

ClassyJacket 2y ago
How are you verifying you're correct? How do you know you're not finding false positives?
- Etheryte 2y ago
  Have you tried reading AI-generated code? Most of the time it's painfully obvious, so long as the snippet isn't short and trivial.
  - thih9 2y ago
    To me it is not obvious. I work with junior level devs and have seen a lot of non-AI junior level code.
    - llmthrow102 2y ago
      You mean, you work with devs who are using AI to generate their code.
      - deleted 2y ago
        [deleted]
      - ben_w 2y ago
        Not saying where, but well before transformers were invented, I saw an iOS project that had huge chunks of uncompiled Symbian code in the project "for reference", an entire pantheon of God classes, entire files duplicated rather than changing access modifiers, 1000 lines inside an always true if block, and 20% of the 120,000 lines were: // And no, those were not generally followed by a real comment.
        tonypace 2y ago
        And yet, I have an unfortunately clear mental picture of the human that did this. In itself, that is a very specific coding style. I don't imagine an LLM would do that. Chat would instead take a couple of the methods from the Symbian codebase and use them where they didn't exist. The God classes would merely be mined for more non-existent functions. The true if block would become a function. And the # lines would have comments on them. Useless comments, but there would be text following every last one of them. Totally different styles.
        ben_w 2y ago
        Depends on the LLM. I've seen exactly what you describe and worse *, and I've also seen them keep to one style until I got bored of prompting for new features to add to the project. * one standard test I have is "make a tetris game as a single page web app", and one model started wrong and then suddenly flipped from Tetris in html/js to ML in python.
      - michaelt 2y ago
        Actually some of us have been in the industry for more than 22 months.
      - max51 2y ago
        I saw a lot of unbelievably bad code when I was teaching in university. I doubt that my undergrad students who couldn't code had access to LLMs in 2011.
acchow 2y ago
> For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' Easy to catch people that aren't trying in the slightest not to get caught, right? I could instead feed a corpus of my own writing to ChatGPT and ask it to write in my style.
- hau 2y ago
  I don't believe it's possible at all if any effort is made beyond prompting chat-like interfaces to "generate X". Given a hand crafted corpus of text even current llms could produce perfect style transfer for a generated continuation. If someone believes it's trivially easy to detect, then they absolutely have no idea what they are dealing with. I assume most people would make least amount of effort and simply prompt chat interface to produce some text, such text is rather detectable. I would like to see some experiments even for this type of detection though.
  - hnlmorg 2y ago
    Are you then plagiarising if the LLM is just regurgitating stuff you’d personally written? The point of these detectors is to spot stuff the students didn’t research and write themselves. But if the corpus is your own written material then you’ve already done the work yourself.
    - throwaway290 2y ago
      LLM is just regurgitating stuff as a principle. You can request someone else's style. People who are easy to detect simply don't do that. But they will learn quickly
      - A4ET8a8uTh0 2y ago
        Yep, some with fun results. I occasionally amuse myself now by asking for X in the style of writing of fictional figure Y. It does have moments.
tessierashpool9 2y ago
the students are too lazy and dumb to do their own thinking and resort to ai. the teachers are also too lazy and dumb to assess the students' work and resort to ai. ain't it funny?
- miningape 2y ago
  It's truly a race to the bottom.
- llmthrow102 2y ago
  To be fair, using humans to spend time sifting through AI slop determining what is and isn't AI generated is not a fight that the humans are going to win.
- A4ET8a8uTh0 2y ago
  I suppose we all get from school what we put into it. I forgot the name of the guy, who said it, but he was some big philosophy lecturer at Harvard and his view on the matter ( heavy reading course and one student left a course review - "not reading assigned reading did not hurt me at all") was ( paraphrased): "This guy is an idiot if he thinks the point of paying $60k a semester of parents money is to sit here and learn nothing.'
  - lupire 2y ago
    He's paying for the degree and the professional network. Studying would be a waste of time.
    - A4ET8a8uTh0 2y ago
      I hope it will not sound too preachy. You are right in a sense that it is what he thinks he is paying for, but is actually missing out on untapped value. He will not be able to discuss death as a concept throughout the lens of various authors. He will not wrestle with questions of cognition and its human limitations ( which amusingly is a relevant subject these days ). He will not learn anything. He is and will remain an adult child in adult daycare. I could go on like this, but I won't. Each of us has a choice how we play the cards we are dealt. I accept your point, but this point reinforces a perspective I heard from my accountant family member, who clearly can identify price, but has a hard time not equating it with value. I hesitate to use the word wrong, because it is pragmatic, but it is also rather wasteful ( if not outright dumb ).
aleph_minus_one 2y ago
> The staff soon realised what garbage these tools are so if a student accused of plagiarism decided to argue back then the accusation would be quietly dropped. I ask myself when the time comes that some student will accuse the stuff of libel or slander becuase of false AI plagiarism accusations.
- red_admiral 2y ago
  Or of racism. There was a thing during the pandemic where automated proctoring tools couldn't cope with people of darker skin than they were trained on; I imagine the first properly verified and scientifically valid examples of AI-detection racism will be found soon.
  - Iulioh 2y ago
    The "dark skin problem" is mostly the camera sensors, not only the training... Low light scenarios are just a thing, you would need more expensive hardware do deal with it.
    - 15155 2y ago
      > mostly the camera sensors Could it be mostly just be..reality? More expensive hardware doesn't somehow make a darker surface reflect more energy in the visible spectrum. "Low light" is not the same condition as "dark surface in well-lit environment." Leaving the visible spectrum is one possible solution, but it's substantially more error-prone and costly. This is still not the same solution as classical CV with "more expensive hardware."
      - red_admiral 2y ago
        If you're building a system to proctor students, then part of your job is to get it to work under all reasonable real-world conditions you might encounter: low light, students with standard webcams or just the one built into their laptop, students with darker skin etc. Reality might make this harder for some cases, but solving that is what you are being paid for. Also, this could have been handled much better in the cases that came up in the media if there had been proper human review of all cases before prosecuting the students.
sumo89 2y ago
My other half is a non-native English speaker. She's fluent but and since ChatGPT came out she's found it very helpful having somewhere to paste a paragraph and get a better version back rather than asking me to rewrite things. That said, she'll often message me with some text and I've got a 100% hit rate for guessing if she's put it through AI first. Once you're used to how they structure sentences it's very easy to spot. I guess the hardest part is being able to prove it if you're in a position of authority like a teacher.
- ben_w 2y ago
  My partner and I are both native English speakers in Germany; if I use ChatGPT to make a sentence in German, he also spots it 100% of the time. (Makes me worry I'm not paying enough attention, that I can't).
- tonypace 2y ago
  It looked like black magic at first. But then you started to see the signs.
- VeninVidiaVicii 2y ago
  Are you guys using free versions of terrible tools? Asking it just to rewrite the whole thing? I use it every day for checking academic figure legends and such, and get extremely minor edits — such as a capitalization or italicization.
p0w3n3d 2y ago
> For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy So far. Unless there is a new generation of teachers who are no longer able to learn on non-AI generated texts because all they get is grammatically corrected by AI for example... Even I am using Grammarly here (as being non-native), but I usually tend to ignore it, because it removes all my "spoken" style, or at least what I think is a "spoken style"
- tonypace 2y ago
  It definitely flattens your style.
JoshTriplett 2y ago
> also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity So do humans. Many people have pet phrases or words that they use unusually often compared to others.
- blitzar 2y ago
  No cap.
- jachee 2y ago
  In the mid 90s (yes I’m dating myself here. :P) I had a classmate who was such a big NIN fan that she worked the phrase “downward spiral” into every single essay she wrote for the entire year.
- pessimizer 2y ago
  People have their favorite phrases or words, but also as readers we fixate on words that we don't personally use, and project that onto the writer. But as a second language learner, you notice that people get stuck on particular words during writing sessions. If I run into a very unusual (and unnecessary) word, I know they're going to use it again within a page or two, maybe once after that, then never again. I blame it on the writer remembering a cool word, or finding a cool word in a thesaurus, then that word dropping out of their active vocabulary after they tried it out a couple times. There's probably an analogue in LLMs, if just because that makes unusual words more likely to repeat themselves in a particular passage.
  - autumnstwilight 2y ago
    I do this when I write, to the point where I have to go back and edit myself after using a slightly unusual word several times in quick succession. I think words I've used recently are easier to access, as if there's a cache for items recently retrieved from deeper layers of memory.
Veen 2y ago
The ones that are easy to spot are easy to spot. You have no idea how much AI-generated work you didn't spot, because you didn't spot it.
xmodem 2y ago
One course I took actually provided students with the output of the plagiarism detector. It was great at correctly identifying where I had directly quoted (and attributed) a source. It would also identify random 5-6 word phrases and attribute them to different random texts on completely different topics where those same 5 words happened to appear.
SilverBirch 2y ago
I did engineering at a university, one of the courses that was mandatory was technical communication. The prof understood that the type of person that went into engineering was not necessarily going to appreciate the subtleties of great literature, so they're course work was extremely rote. It was like "Write about a technical subject, doesn't matter what, 1500 words, here's the exact score card". And the score card was like "Uses a sentence to introduce the topic of the paragraph". The result was that you write extremely formulaic prose. Now, I'm not sure that was going to teach people to ever be great communicators, but I think it worked extremely well to bring someone who communicated very badly up to some basic minimum standard. It could be extremely effective applied to the (few) other courseworks that required prose too - partly because by being so formulaic you appealed the overworked PhD student who was likely marking it. It seems likely that a suitably disciplined student could look a lot like ChatGPT and the cost of a false accusation is extremely high.
- VeninVidiaVicii 2y ago
  This is my exact issue. ChatGPT seems formulaic in part, because so much of the work it’s trained on is also formulaic or at least predictable.
- jjmarr 2y ago
  Extremely disciplined students always feed papers into AI detectors before submitting and then revise their work until it passes. Dodging the detector is done regardless of whether or not one has used AI to write that paper.
wrasee 2y ago
> trivially easy That’s the problem. It is trivially easy, 99% of the time. But that misses the entire point of the article. If I got 99% on an exam I’d say that was trivially easy. But making one mistake in a hundred is not ok when it’s someone else’s livelihood.
shusaku 2y ago
What are you asking your applicants to do that LLM use is a problem? I see no issue with having a machine compile one’s history into a resume. Is their purpose statement not original enough /s?
Buttons840 2y ago
Students who use the "word of the week" can easily explain it by saying they used an AI in their studies. "You asked us to write an essay on the Civil War. The first thing I did was ask an AI to explain it to me, and I asked the AI some follow-up questions. Then I did some research using other sources and wrote my paper." It might even be a true story, and in such a case it's not surprising that the student would repeat words they encountered while studying.
rahimnathwani 2y ago
For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. When evaluating job applications we don't have ground truth labels, so we cannot possibly know the precision or recall of our classification.

ClassyJacket 2y ago

How are you verifying you're correct? How do you know you're not finding false positives?

Etheryte 2y ago
Have you tried reading AI-generated code? Most of the time it's painfully obvious, so long as the snippet isn't short and trivial.
- thih9 2y ago
  To me it is not obvious. I work with junior level devs and have seen a lot of non-AI junior level code.
  - llmthrow102 2y ago
    You mean, you work with devs who are using AI to generate their code.
    - deleted 2y ago
      [deleted]
    - ben_w 2y ago
      Not saying where, but well before transformers were invented, I saw an iOS project that had huge chunks of uncompiled Symbian code in the project "for reference", an entire pantheon of God classes, entire files duplicated rather than changing access modifiers, 1000 lines inside an always true if block, and 20% of the 120,000 lines were: // And no, those were not generally followed by a real comment.
      - tonypace 2y ago
        And yet, I have an unfortunately clear mental picture of the human that did this. In itself, that is a very specific coding style. I don't imagine an LLM would do that. Chat would instead take a couple of the methods from the Symbian codebase and use them where they didn't exist. The God classes would merely be mined for more non-existent functions. The true if block would become a function. And the # lines would have comments on them. Useless comments, but there would be text following every last one of them. Totally different styles.
        ben_w 2y ago
        Depends on the LLM. I've seen exactly what you describe and worse *, and I've also seen them keep to one style until I got bored of prompting for new features to add to the project. * one standard test I have is "make a tetris game as a single page web app", and one model started wrong and then suddenly flipped from Tetris in html/js to ML in python.
    - michaelt 2y ago
      Actually some of us have been in the industry for more than 22 months.
    - max51 2y ago
      I saw a lot of unbelievably bad code when I was teaching in university. I doubt that my undergrad students who couldn't code had access to LLMs in 2011.

acchow 2y ago

> For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' Easy to catch people that aren't trying in the slightest not to get caught, right? I could instead feed a corpus of my own writing to ChatGPT and ask it to write in my style.

hau 2y ago
I don't believe it's possible at all if any effort is made beyond prompting chat-like interfaces to "generate X". Given a hand crafted corpus of text even current llms could produce perfect style transfer for a generated continuation. If someone believes it's trivially easy to detect, then they absolutely have no idea what they are dealing with. I assume most people would make least amount of effort and simply prompt chat interface to produce some text, such text is rather detectable. I would like to see some experiments even for this type of detection though.
- hnlmorg 2y ago
  Are you then plagiarising if the LLM is just regurgitating stuff you’d personally written? The point of these detectors is to spot stuff the students didn’t research and write themselves. But if the corpus is your own written material then you’ve already done the work yourself.
  - throwaway290 2y ago
    LLM is just regurgitating stuff as a principle. You can request someone else's style. People who are easy to detect simply don't do that. But they will learn quickly
    - A4ET8a8uTh0 2y ago
      Yep, some with fun results. I occasionally amuse myself now by asking for X in the style of writing of fictional figure Y. It does have moments.

tessierashpool9 2y ago

the students are too lazy and dumb to do their own thinking and resort to ai. the teachers are also too lazy and dumb to assess the students' work and resort to ai. ain't it funny?

miningape 2y ago
It's truly a race to the bottom.
llmthrow102 2y ago
To be fair, using humans to spend time sifting through AI slop determining what is and isn't AI generated is not a fight that the humans are going to win.
A4ET8a8uTh0 2y ago
I suppose we all get from school what we put into it. I forgot the name of the guy, who said it, but he was some big philosophy lecturer at Harvard and his view on the matter ( heavy reading course and one student left a course review - "not reading assigned reading did not hurt me at all") was ( paraphrased): "This guy is an idiot if he thinks the point of paying $60k a semester of parents money is to sit here and learn nothing.'
- lupire 2y ago
  He's paying for the degree and the professional network. Studying would be a waste of time.
  - A4ET8a8uTh0 2y ago
    I hope it will not sound too preachy. You are right in a sense that it is what he thinks he is paying for, but is actually missing out on untapped value. He will not be able to discuss death as a concept throughout the lens of various authors. He will not wrestle with questions of cognition and its human limitations ( which amusingly is a relevant subject these days ). He will not learn anything. He is and will remain an adult child in adult daycare. I could go on like this, but I won't. Each of us has a choice how we play the cards we are dealt. I accept your point, but this point reinforces a perspective I heard from my accountant family member, who clearly can identify price, but has a hard time not equating it with value. I hesitate to use the word wrong, because it is pragmatic, but it is also rather wasteful ( if not outright dumb ).

aleph_minus_one 2y ago

> The staff soon realised what garbage these tools are so if a student accused of plagiarism decided to argue back then the accusation would be quietly dropped. I ask myself when the time comes that some student will accuse the stuff of libel or slander becuase of false AI plagiarism accusations.

red_admiral 2y ago
Or of racism. There was a thing during the pandemic where automated proctoring tools couldn't cope with people of darker skin than they were trained on; I imagine the first properly verified and scientifically valid examples of AI-detection racism will be found soon.
- Iulioh 2y ago
  The "dark skin problem" is mostly the camera sensors, not only the training... Low light scenarios are just a thing, you would need more expensive hardware do deal with it.
  - 15155 2y ago
    > mostly the camera sensors Could it be mostly just be..reality? More expensive hardware doesn't somehow make a darker surface reflect more energy in the visible spectrum. "Low light" is not the same condition as "dark surface in well-lit environment." Leaving the visible spectrum is one possible solution, but it's substantially more error-prone and costly. This is still not the same solution as classical CV with "more expensive hardware."
    - red_admiral 2y ago
      If you're building a system to proctor students, then part of your job is to get it to work under all reasonable real-world conditions you might encounter: low light, students with standard webcams or just the one built into their laptop, students with darker skin etc. Reality might make this harder for some cases, but solving that is what you are being paid for. Also, this could have been handled much better in the cases that came up in the media if there had been proper human review of all cases before prosecuting the students.

sumo89 2y ago

My other half is a non-native English speaker. She's fluent but and since ChatGPT came out she's found it very helpful having somewhere to paste a paragraph and get a better version back rather than asking me to rewrite things. That said, she'll often message me with some text and I've got a 100% hit rate for guessing if she's put it through AI first. Once you're used to how they structure sentences it's very easy to spot. I guess the hardest part is being able to prove it if you're in a position of authority like a teacher.

ben_w 2y ago
My partner and I are both native English speakers in Germany; if I use ChatGPT to make a sentence in German, he also spots it 100% of the time. (Makes me worry I'm not paying enough attention, that I can't).
tonypace 2y ago
It looked like black magic at first. But then you started to see the signs.
VeninVidiaVicii 2y ago
Are you guys using free versions of terrible tools? Asking it just to rewrite the whole thing? I use it every day for checking academic figure legends and such, and get extremely minor edits — such as a capitalization or italicization.

p0w3n3d 2y ago

> For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy So far. Unless there is a new generation of teachers who are no longer able to learn on non-AI generated texts because all they get is grammatically corrected by AI for example... Even I am using Grammarly here (as being non-native), but I usually tend to ignore it, because it removes all my "spoken" style, or at least what I think is a "spoken style"

tonypace 2y ago
It definitely flattens your style.

JoshTriplett 2y ago

> also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity So do humans. Many people have pet phrases or words that they use unusually often compared to others.

blitzar 2y ago
No cap.
jachee 2y ago
In the mid 90s (yes I’m dating myself here. :P) I had a classmate who was such a big NIN fan that she worked the phrase “downward spiral” into every single essay she wrote for the entire year.
pessimizer 2y ago
People have their favorite phrases or words, but also as readers we fixate on words that we don't personally use, and project that onto the writer. But as a second language learner, you notice that people get stuck on particular words during writing sessions. If I run into a very unusual (and unnecessary) word, I know they're going to use it again within a page or two, maybe once after that, then never again. I blame it on the writer remembering a cool word, or finding a cool word in a thesaurus, then that word dropping out of their active vocabulary after they tried it out a couple times. There's probably an analogue in LLMs, if just because that makes unusual words more likely to repeat themselves in a particular passage.
- autumnstwilight 2y ago
  I do this when I write, to the point where I have to go back and edit myself after using a slightly unusual word several times in quick succession. I think words I've used recently are easier to access, as if there's a cache for items recently retrieved from deeper layers of memory.

Veen 2y ago

The ones that are easy to spot are easy to spot. You have no idea how much AI-generated work you didn't spot, because you didn't spot it.

xmodem 2y ago

One course I took actually provided students with the output of the plagiarism detector. It was great at correctly identifying where I had directly quoted (and attributed) a source. It would also identify random 5-6 word phrases and attribute them to different random texts on completely different topics where those same 5 words happened to appear.

SilverBirch 2y ago

I did engineering at a university, one of the courses that was mandatory was technical communication. The prof understood that the type of person that went into engineering was not necessarily going to appreciate the subtleties of great literature, so they're course work was extremely rote. It was like "Write about a technical subject, doesn't matter what, 1500 words, here's the exact score card". And the score card was like "Uses a sentence to introduce the topic of the paragraph". The result was that you write extremely formulaic prose. Now, I'm not sure that was going to teach people to ever be great communicators, but I think it worked extremely well to bring someone who communicated very badly up to some basic minimum standard. It could be extremely effective applied to the (few) other courseworks that required prose too - partly because by being so formulaic you appealed the overworked PhD student who was likely marking it. It seems likely that a suitably disciplined student could look a lot like ChatGPT and the cost of a false accusation is extremely high.

VeninVidiaVicii 2y ago
This is my exact issue. ChatGPT seems formulaic in part, because so much of the work it’s trained on is also formulaic or at least predictable.
jjmarr 2y ago
Extremely disciplined students always feed papers into AI detectors before submitting and then revise their work until it passes. Dodging the detector is done regardless of whether or not one has used AI to write that paper.

wrasee 2y ago

> trivially easy That’s the problem. It is trivially easy, 99% of the time. But that misses the entire point of the article. If I got 99% on an exam I’d say that was trivially easy. But making one mistake in a hundred is not ok when it’s someone else’s livelihood.

shusaku 2y ago

What are you asking your applicants to do that LLM use is a problem? I see no issue with having a machine compile one’s history into a resume. Is their purpose statement not original enough /s?

Buttons840 2y ago

Students who use the "word of the week" can easily explain it by saying they used an AI in their studies. "You asked us to write an essay on the Civil War. The first thing I did was ask an AI to explain it to me, and I asked the AI some follow-up questions. Then I did some research using other sources and wrote my paper." It might even be a true story, and in such a case it's not surprising that the student would repeat words they encountered while studying.

rahimnathwani 2y ago

For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. When evaluating job applications we don't have ground truth labels, so we cannot possibly know the precision or recall of our classification.