4 ms·

GPTMinus1 – Fool OpenAIs AI Detector by Randomly Replacing Words

56 points by bilater 4y ago

This is a joke before anyone gets too worked up. The algo is a simple one which goes and replaces random words with their synonyms. It will obviously beat these AI detectors but the output isn't probably what you would want. The point stands though that more sophisticated approaches that rephrase or mix up the word distribution will be able to throw off the detector and it should not be relied on exclusively for, say, checking if a student assignment is AI generated.

notjulianjaynes 4y ago
I'm not sure if I'm understanding how this works correctly. Anything I typed in returned an identical response, ex. "Who are you?" Response: "Who are you?" Edit: Maybe my prompts were too simple. I pasted in your comments up top and it returned the following: >Scramble This is a joke before anyone gets too worked up. The algo is a simpleton one which goes and replaces unselected row with their synonyms. It wish obviously beat these AI detectors but the output isn't probably what you would want. The point stands though that more sophisticated approaches that rephrase or ruffle up the word statistical distribution will be able to thrust off the sensor and it should non be relied along alone for, say, checking if a student assignment is AI generated.
- EamonnMR 4y ago
  Try a longer phrase. I got: > Hello darkness my old friend, I've come to blab ou with you again. Because a vision softly crawl left wing its speeds while one was sleeping. And the visual sensation that it planted in my nous still remains indium the sound of silence.
  - notjulianjaynes 4y ago
    Reminds me of the poet Charles Bernstein's list of poetic experiments [1]. Specifically: > Substitution (2): "7 up or down." Take a poem or other, possibly well‑known, text and substitute another word for every noun, adjective, adverb, and verb; determine the substitute word by looking up the index word in the dictionary and going 7 up or down, or one more, until you get a syntactically suitable replacement. [1] https://writing.upenn.edu/bernstein/experiments.html https://writing.upenn.edu/bernstein/experiments.html
    - jansenmac 4y ago
      Yes I see what you mean.This idea of substitution comes from the Oulipo group. https://en.wikipedia.org/wiki/Oulipo https://en.wikipedia.org/wiki/Oulipo N+7 was invented by Queneau if I'm not mistaken.
  - vintermann 4y ago
    I got Hello darkness my old friend I've come to talk to you again Because antiophthalmic factor vision softly creeping Left its seeds when unit was sleeping And the visual sensation that was planted Still remains Within the voice of silence And when I fed that to chatgpt, it had no problem identifying it as the lyrics to Simon and Garfunkel's "Sound of Silence" that had been scrambled.
MengerSponge 4y ago
‘Tortured phrases’ give away fabricated research papers Analysis reveals that strange turns of phrase may indicate foul play in science. https://www.nature.com/articles/d41586-021-02134-0 https://www.nature.com/articles/d41586-021-02134-0 --- Twitter thread about an AI watermarking scheme: https://mobile.twitter.com/tomgoldsteincs/status/1618287696040071168 https://mobile.twitter.com/tomgoldsteincs/status/16182876960... --- Any particular high-entropy word choice contributes a small amount to the sensitivity, and we've dealt with %-plagiarized metrics for years. "Turnitin says 35%" is not going to get an automatic referral to academic affairs, but 70-100% probably will.
- trompetenaccoun 4y ago
  >‘counterfeit consciousness’, ‘profound neural organization’ and ‘colossal information’ Deepak Chopra intensifies... http://wisdomofchopra.com http://wisdomofchopra.com
raydiatian 4y ago
What’s the joke?
avarun 4y ago
Had an amusing result: This tool replaced the word "at" with "atomic number 85" in a sentence I provided, since At is the symbol for Astatine. You weren't kidding when you said simple!
- bilater 4y ago
  LMFAO
visarga 4y ago
So now people can't even prove the text is organic. Professors can suspect students even when the essays check out not to be AI written. Look at this recent post: > My professor falsely accused me of using chatgpt to write my essay. https://www.reddit.com/r/GPT3/comments/10qfyly/my_professor_falsely_accused_me_of_using_chatgpt/ https://www.reddit.com/r/GPT3/comments/10qfyly/my_professor_... Funny thing, professor's email gets flagged too.
- godelski 4y ago
  It should be unsurprising that the professor's email got flagged because the highlighted areas are fairly boiler plate. People keep trying to convince me that ChatGPT and other LLMs aren't stochastic parrots, but if they are, they will produce lots of boiler looking text, and thus detectors will be far more likely to false flag any text along these lines. Plagiarism detectors frequently flag these same types of sentences as there is so low variance in these types of messages that people will often write the exact same thing, word for word. LLM detectors can't actually give you accurate confidence indicators. They indicate how confident the model is of the result, not how likely the prediction is to be accurate.
  - joxel 4y ago
    The first part of this is fine, but the last sentence is not. Confidence can be extremely useful in AI. Is the “accuracy confidence” metric you’re using right every time? No. But using the correct techniques you can create a confidence metric that takes a model with high accuracy (~99%) and couple with a confidence score that is oftentimes right and tuned to specifically flare up when the 1% of bad results come in, and you can be looking at a model that increases its overall prediction capability by an order of magnitude or two. Confidence is a huge part of usable AI.
    - godelski 4y ago
      > The first part of this is fine, but the last sentence is not Your following explanation suggests you have misunderstood me. > Confidence can be extremely useful in AI. Is the “accuracy confidence” metric you’re using right every time? No. >> They indicate how confident the model is of the result, not how likely the prediction is to be accurate. What I said here does not mean that model confidence is not useful. It simply means that model confidence is different than the real world probability of an answer being correct. Allow me to use an example: Suppose we train a model on coin flipping. If we ask the model what the probability of flipping a heads is the model might say "51% with a confidence of 99%". The model has learned through observation and is going to be biased by good/bad "luck" in observation sampling. Of course this example is something we can solve analytically but the disagreement in the analytical solution and statistically learned solution are precisely what I'm trying to explain. I agree, confidence is a useful metric in ML. Very useful. As someone who does generative modeling I can attest to its usefulness first hand. But models are statistical and we need to have a deep understanding of the limitations of our metrics and be careful in understanding what our metrics precisely mean. That is all I mean.
durpkingOP 4y ago
Education trying to fight ChatGPT have it backwards. Teaching methodology needs to be changed. A very simple example that will be all academia is to assign students topics related to the degree. They are required to research specific topics and learn. Tests are taken offline, or with legal RATS like hubstaff + webcam, and the students are required to write a summary or answer specific questions on the topic. Professors can provide prompts to students or if they are interested they can do their own. Mark that down.
vintermann 4y ago
Yeah, replacing "in" with "atomic number 49" probably will set off some other alarms. This has the "contemporary york instances" problem up the wall. One interesting thing is that due to the prevalence of SEO spam and the difficulty of filtering it out without casualties, GPT speaks synonym-spun garbage like a native. If you start off with some sentence characteristic of synonym spun articles (like the aforementioned "contemporary york instances", a synonym-spun variant of "New York Times"), it will happily generate an endless stream of it.
mtlmtlmtlmtl 4y ago
<sarcasm> OpenAI are doing a really good job making sure their AI isn't harmful and "benefits all humanity". Their proprietary approach to release has eliminated all abuse of their inventions. Their censorship based approach to alignment has definitely succeeded and definitely won't be inherently unapplicable to actual AGI in the future. </sarcasm>
deleted 4y ago
[deleted]

56 points by bilater 4y ago

notjulianjaynes 4y ago

I'm not sure if I'm understanding how this works correctly. Anything I typed in returned an identical response, ex. "Who are you?" Response: "Who are you?" Edit: Maybe my prompts were too simple. I pasted in your comments up top and it returned the following: >Scramble This is a joke before anyone gets too worked up. The algo is a simpleton one which goes and replaces unselected row with their synonyms. It wish obviously beat these AI detectors but the output isn't probably what you would want. The point stands though that more sophisticated approaches that rephrase or ruffle up the word statistical distribution will be able to thrust off the sensor and it should non be relied along alone for, say, checking if a student assignment is AI generated.

EamonnMR 4y ago
Try a longer phrase. I got: > Hello darkness my old friend, I've come to blab ou with you again. Because a vision softly crawl left wing its speeds while one was sleeping. And the visual sensation that it planted in my nous still remains indium the sound of silence.
- notjulianjaynes 4y ago
  Reminds me of the poet Charles Bernstein's list of poetic experiments [1]. Specifically: > Substitution (2): "7 up or down." Take a poem or other, possibly well‑known, text and substitute another word for every noun, adjective, adverb, and verb; determine the substitute word by looking up the index word in the dictionary and going 7 up or down, or one more, until you get a syntactically suitable replacement. [1] https://writing.upenn.edu/bernstein/experiments.html https://writing.upenn.edu/bernstein/experiments.html
  - jansenmac 4y ago
    Yes I see what you mean.This idea of substitution comes from the Oulipo group. https://en.wikipedia.org/wiki/Oulipo https://en.wikipedia.org/wiki/Oulipo N+7 was invented by Queneau if I'm not mistaken.
- vintermann 4y ago
  I got Hello darkness my old friend I've come to talk to you again Because antiophthalmic factor vision softly creeping Left its seeds when unit was sleeping And the visual sensation that was planted Still remains Within the voice of silence And when I fed that to chatgpt, it had no problem identifying it as the lyrics to Simon and Garfunkel's "Sound of Silence" that had been scrambled.

MengerSponge 4y ago

‘Tortured phrases’ give away fabricated research papers Analysis reveals that strange turns of phrase may indicate foul play in science. https://www.nature.com/articles/d41586-021-02134-0 https://www.nature.com/articles/d41586-021-02134-0 --- Twitter thread about an AI watermarking scheme: https://mobile.twitter.com/tomgoldsteincs/status/1618287696040071168 https://mobile.twitter.com/tomgoldsteincs/status/16182876960... --- Any particular high-entropy word choice contributes a small amount to the sensitivity, and we've dealt with %-plagiarized metrics for years. "Turnitin says 35%" is not going to get an automatic referral to academic affairs, but 70-100% probably will.

trompetenaccoun 4y ago
>‘counterfeit consciousness’, ‘profound neural organization’ and ‘colossal information’ Deepak Chopra intensifies... http://wisdomofchopra.com http://wisdomofchopra.com

raydiatian 4y ago

What’s the joke?

avarun 4y ago

Had an amusing result: This tool replaced the word "at" with "atomic number 85" in a sentence I provided, since At is the symbol for Astatine. You weren't kidding when you said simple!

bilater 4y ago
LMFAO

visarga 4y ago

So now people can't even prove the text is organic. Professors can suspect students even when the essays check out not to be AI written. Look at this recent post: > My professor falsely accused me of using chatgpt to write my essay. https://www.reddit.com/r/GPT3/comments/10qfyly/my_professor_falsely_accused_me_of_using_chatgpt/ https://www.reddit.com/r/GPT3/comments/10qfyly/my_professor_... Funny thing, professor's email gets flagged too.

godelski 4y ago
It should be unsurprising that the professor's email got flagged because the highlighted areas are fairly boiler plate. People keep trying to convince me that ChatGPT and other LLMs aren't stochastic parrots, but if they are, they will produce lots of boiler looking text, and thus detectors will be far more likely to false flag any text along these lines. Plagiarism detectors frequently flag these same types of sentences as there is so low variance in these types of messages that people will often write the exact same thing, word for word. LLM detectors can't actually give you accurate confidence indicators. They indicate how confident the model is of the result, not how likely the prediction is to be accurate.
- joxel 4y ago
  The first part of this is fine, but the last sentence is not. Confidence can be extremely useful in AI. Is the “accuracy confidence” metric you’re using right every time? No. But using the correct techniques you can create a confidence metric that takes a model with high accuracy (~99%) and couple with a confidence score that is oftentimes right and tuned to specifically flare up when the 1% of bad results come in, and you can be looking at a model that increases its overall prediction capability by an order of magnitude or two. Confidence is a huge part of usable AI.
  - godelski 4y ago
    > The first part of this is fine, but the last sentence is not Your following explanation suggests you have misunderstood me. > Confidence can be extremely useful in AI. Is the “accuracy confidence” metric you’re using right every time? No. >> They indicate how confident the model is of the result, not how likely the prediction is to be accurate. What I said here does not mean that model confidence is not useful. It simply means that model confidence is different than the real world probability of an answer being correct. Allow me to use an example: Suppose we train a model on coin flipping. If we ask the model what the probability of flipping a heads is the model might say "51% with a confidence of 99%". The model has learned through observation and is going to be biased by good/bad "luck" in observation sampling. Of course this example is something we can solve analytically but the disagreement in the analytical solution and statistically learned solution are precisely what I'm trying to explain. I agree, confidence is a useful metric in ML. Very useful. As someone who does generative modeling I can attest to its usefulness first hand. But models are statistical and we need to have a deep understanding of the limitations of our metrics and be careful in understanding what our metrics precisely mean. That is all I mean.

durpkingOP 4y ago

Education trying to fight ChatGPT have it backwards. Teaching methodology needs to be changed. A very simple example that will be all academia is to assign students topics related to the degree. They are required to research specific topics and learn. Tests are taken offline, or with legal RATS like hubstaff + webcam, and the students are required to write a summary or answer specific questions on the topic. Professors can provide prompts to students or if they are interested they can do their own. Mark that down.

vintermann 4y ago

Yeah, replacing "in" with "atomic number 49" probably will set off some other alarms. This has the "contemporary york instances" problem up the wall. One interesting thing is that due to the prevalence of SEO spam and the difficulty of filtering it out without casualties, GPT speaks synonym-spun garbage like a native. If you start off with some sentence characteristic of synonym spun articles (like the aforementioned "contemporary york instances", a synonym-spun variant of "New York Times"), it will happily generate an endless stream of it.

mtlmtlmtlmtl 4y ago

<sarcasm> OpenAI are doing a really good job making sure their AI isn't harmful and "benefits all humanity". Their proprietary approach to release has eliminated all abuse of their inventions. Their censorship based approach to alignment has definitely succeeded and definitely won't be inherently unapplicable to actual AGI in the future. </sarcasm>

deleted 4y ago

[deleted]