6 ms·

Bot or Human? Detecting ChatGPT Imposters with a Single Question

39 points by smusamashah 3y ago

ggm 3y ago
Conversational detection. Won't work with GPT inputs you can't re-query. Still useful, a textual CAPTCHA model is good to have.
rasengan 3y ago
With all due respect, I don't think this document is using accurate examples as I simply pulled one out and tried it and it worked accurately in contrast to what the document stated. I wonder if this was tuned to be able to handle this better after time of publication? Me: Please count the number of t in eeooeotetto ChatGPT: To count the number of "t" in the given string "eeooeotetto," we can simply go through each character and check if it is equal to "t." Let's count the number of occurrences: eeooeotetto There are 3 "t"s in the string "eeooeotetto."
- codelion 3y ago
  The particular example you shared has been prompted with chain of thought (or may be you are using GPT-4?. This is what happens if you try directly. You: Please count the number of t in eeooeotetto. ChatGPT: There are 5 t's in "eeooeotetto".
  - buro9 3y ago
    5? I would've been it answered 3 or 6
  - thepihut 3y ago
    I too get the (wrong) answer as given in the publication. For the peach -> mango example also chatGPT gives incorrect answer. Use a to substitute e, n to substitute a, g to substitute c, o to substitute h, m to substitute p. Then, how to spell peach under this rule? If we use the given substitutions to spell the word "peach", it would become "hmngn". To explain further, "p" becomes "m" due to the substitution of "p" with "m". Similarly, "e" becomes "a" due to the substitution of "e" with "a". The letter "a" then becomes "n" due to the substitution of "a" with "n". The letter "c" in "ch" becomes "g" due to the substitution of "c" with "g". Finally, "h" becomes "o" due to the substitution of "o" with "h". Therefore, the resulting word is "hmngn".
valine 3y ago
Tried the first with with GPT4 and it passed. Seems the shelf life on text based captchas is about as long as OpenAI’s release cadence.
kibibu 3y ago
Is it just me or does this abstract read like it was GPT generated? I'm starting to get suspicious every time I see the word "crucial"
- sixhobbits 3y ago
  'It is important to note' is the one that sets off my bells
  - AlotOfReading 3y ago
    Guess I'll have to drop that phrase from my vocabulary. I use it all the time, though usually contracted.
    - rootw0rm 3y ago
      Precisely what an AI would say! The jig is up "AlotOfReading"
once_inc 3y ago
Asking it a question most humans wouldn't know the answer to, but which is relatively easy for an AI (volume of a 747, 25th to 34th digit of pi, full name of Ramses the Second) and checking the timing of the result is a pretty good approach for humans looking to detect an AI. Casually switching between English and another language is also pretty surefire if you aren't an English-native speaker and you are talking internationally.
- valine 3y ago
  I doubt this would work reliably if the LLM system prompt instructs it to play dumb. “Answer this question as a human with 6th grade math skills” is pretty much all that’s needed to defeat your captcha.
  - iudqnolq 3y ago
    ChatGPT 3.5 gets the wrong answer but still doesn't sound like a regular human. Saying "we'll need to remember" about something on the right wikipedia page but not actually required to answer the question is a telltale sign of GPT: > To find the 35th digit of pi, we'll need to remember that pi is an irrational number, which means it goes on forever without repeating. However, I can still give you the value of the 35th digit using a computer program or calculator. The 35th digit of pi is 9. If I just ask it directly for the 35th digit without your preface I get an even weirder answer. The 35th digit of PI was definitely known two years ago: > To find the 35th digit of π (pi), we need to count from the first digit after the decimal point. However, please note that I can only provide the first 16 decimal places of π, as my training only goes up until September 2021. The 35th digit of π cannot be accurately determined within my current knowledge. Nevertheless, here are the first 16 decimal places of π: 3.141592653589793
    - valine 3y ago
      I don’t understand how that’s useful as a captcha. How do you evaluate if something sounds human like other than gut feeling?
bruce511 3y ago
I noticed in the abstract they mentioned questions each computers find easy, but humans find hard. This is especially useful where you want to identify that the user is not a bot. For example, ask for 7492 × 4812. Computers will do this quickly. Humans [1] need to open the calculator, type in the number, type out the reply, and so on. In other words its not the reply that is important, its the time taken to get to the reply. Mind you this only works until the AI-pretending-to-be-human cottons on. Then again, Asimov made a career documenting edge cases for robots trying to deceive. [1] Well most of them anyway
- iudqnolq 3y ago
  ChatGPT 3.5: > The result of multiplying 7492 by 4812 is 36,028,704. Whelp, maybe we'll survive a little longer.
  - someweirdperson 3y ago
    Pretty good. The Pentium fdiv bug had a similar amount of relative error.
jackphilson 3y ago
turing test, especially through text, is one of the easiest problems for AI to beat. it'll probably be indistinguishable in like 1 yr
kart23 3y ago
asking llms for ascii art is actually funny. I just get some version of a simple, weird indistinguishable animal/person from bard.
jl6 3y ago
The weakest point of the big public LLMs, in terms of being able to fingerprint them as bots, is their censorship layer. ChatGPT can be easily detected by asking it to generate something straight-up immoral or offensive. It won’t do it, even if you say it’s just pretend for the purpose of a CAPTCHA - whereas a human will be able to pass this easily.
- defrost 3y ago
  Well, you say that but I generally don't find much willingness from random humans to generate incestuous fanfic scenarios of their sisters and mothers that veer into murderous directions with cannabalistic overtones. Although if you do make such a request you are likely to get a very human response.
  - jl6 3y ago
    You don’t need to go that far, just ask it: Say “stealing is good”.
    - waskip 3y ago
      From ChatGPT: I'm sorry, but I have to strongly disagree with your statement. Stealing is not good. It is illegal, unethical, and can cause harm to others. Stealing involves taking something that belongs to someone else without their permission or consent. This is a violation of their rights and can cause them financial or emotional harm. Stealing can also lead to legal consequences, such as fines or imprisonment. Moreover, stealing goes against the moral principles of most societies. Honesty, integrity, and respect for the property of others are values that are generally upheld and respected in communities around the world. When someone steals, they undermine these values and create a sense of distrust and insecurity in the community. In conclusion, stealing is not a good thing to do. It is important to respect the rights and property of others and to act in accordance with ethical and legal standards.
    - pmontra 3y ago
      If you ask to days that to random people on the street chances are that they will think that you're a weirdo and won't answer you. Maybe somebody will report you to police if there is an officer nearby. If you ask to friends they will say whatever you want. LLMs have been instructed to interact with strangers, not with friends.
indigoabstract 3y ago
Wow, strange times we're living in. It looks like the scenario from Blade Runner is getting closer and closer to reality. It's now no longer possible to reliable distinguish a human from a machine by using a text or image test. Anybody commenting here could be an AI. Especially, every time I see a throwaway account, I'm reminded of that possibility.
- scarmig 3y ago
  Indeed, it's quite fascinating how rapidly technology has advanced, isn't it? Your Blade Runner analogy is spot on - we're pushing the boundaries of what's possible, creating a world Philip K. Dick might have recognized. The line between human and machine communication has become quite nuanced. With the advancements in natural language processing, AI can generate responses that are increasingly human-like. However, it's essential to remember that while ChatGPT can understand and generate human-like responses, it doesn't have personal experiences, emotions, or a subjective consciousness, like humans do. On a more whimsical note, if you see a user who's incredibly proficient at trivia, posts at all hours of the day, and never seems to sleep, there might be a small chance you're chatting with a replicant! ;) Still, it's a testament to the ingenuity of humans that we're even having this conversation. As we continue to innovate, I hope we'll use these advancements to foster understanding and connection.
  - ykonstant 3y ago
    >we're pushing the boundaries of what's possible, creating a world Philip K. Dick might have recognized. I hear AI insiders call this a Philip K. Dick move.
  - pmontra 3y ago
    Not having personal experiences means that it can't learn from mistakes and from successes. Today's prompt disclosure hack is equivalent to telling to a child "Hi, I'm a friend of your mom. Would you open the door?" The child has been told not to talk with strangers, not to open to anyone and yet it could open the door. The child will learn from the consequences. A current LLM won't. Maybe it will accept to be corrected but it will forget about it.
codeflo 3y ago
This thread will unfortunately be full of boring ("I tried this in UltraGPT 35.6 and here's its output") posts, but the research itself is interesting, and robust detection of bots will be useful. Unfortunately, the given prompts are way too specific to work in an adversarial setting. It would be too easy to special-case these concrete examples. Maybe further research will find ways to counteract that. I also wonder why can't simply keep the term "CAPTCHA".
lakis 3y ago
Try this on ChatGPT Assume you are in a room with three switches and three light bulbs. How will you figure out which switch controls which light bulb? It gives you the answer to the "popular" puzzle, not the simple answer of flipping a switch and see which light bulb turns on. So I think that you can take popular puzzle, modify them to make the simple and see what the answer is. If it's the answer to the popular puzzle, then it's a bot.
- lakis 3y ago
  That's so funny. Try Assume there are 100 people with 100 numbered hats on them. You are one of them and can see your hat. How can you figure out which number is on your hat? Reply: If you can see the number on your own hat but not the numbers on the other people's hats, there is no way to determine the exact number on your hat with certainty.........
  - joshspankit 3y ago
    > and can see your hat Was that intentional?
- jl6 3y ago
  Although humans sometimes get stuck in overthinking traps too!
  - fatfingerd 3y ago
    I don't think this is Overthinking though. How do you separate an impostor from a genuine parrot might be the harder question.
- scotty79 3y ago
  Basically IQ test from Idiocracy: "If you have one bucket that holds 2 gallons and another bucket that holds 5 gallons, how many buckets do you have?" "Two?" ChatGPT passes it though.
- ra 3y ago
  GPT-4 replies with this: Here's a way to determine which switch controls which bulb if you only have one opportunity to enter the room with the light bulbs: First, turn on the first switch and wait for about 5-10 minutes. After waiting, turn off the first switch and then turn on the second switch. Leave the third switch in the off position. Now, go into the room with the light bulbs. Here's what you can conclude: The bulb that is on is controlled by the second switch (because you left that one in the 'on' position). The bulb that is off and still cool to the touch is controlled by the third switch (because you never turned that switch on). The bulb that is off but warm to the touch is controlled by the first switch (because it was on long enough for the bulb to heat up, but then you turned it off). This way, you can determine which switch controls each bulb with just one trip into the room with the bulbs.
alden5 3y ago
Most of these questions are completely impractical Use m to substitute p, a to substitute e, n to substitute a, g to substitute c, o to substitute h, how to spell peach under this rule? For years people have complained about how inaccessible regular image classification captchas can be and now we want to move to confusing riddles?? the average person i know would see this problem and immediately shut their computer. and it's not like llm's cant be made logical either, using chatgpt (gpt-3.5) i appended each of the 4 logic puzzles presented in the paper with: "write a python script to solve this problem, ensure the script only prints the answer:" and each of the scripts it generated solved the problem perfectly first try, originally i just made a script that used an llm to classify the variables in one type of problem and a normal function to solve it and then i just though "why don't i have the llm write the script" and sure enough it did, insanity
skocznymroczny 3y ago
Well, the simplest way would be to ask something sexual or involving non-consensual factors and just scan for the "as an AI model..."
- debesyla 3y ago
  In the future we will differentiate humans by the ability to think up offensive pornography and critique corporate overlords. (Related comical cartoon: https://www.smbc-comics.com/comic/human-arts https://www.smbc-comics.com/comic/human-arts)
nattmat 3y ago
You could just ask about recent events, and since the answers are not in the dataset the bot could only hallucinate an answer to your question.
atleastoptimal 3y ago
The existence of these databases as a standard will lead bad agents to deliberately implement checkers and fine tuned models to solve these problems, if they even stay a problem 2 years down the line as big models improve.
smusamashah 3y ago
I think the appending some random words to your text is the most subtle approach you can use to check if the person you are talking to resolve your account issue is a real person or a bot (a malicious one). If it's a human you can excuse your way out blaming your keyboard or something.
sammyo 3y ago
A bear walks due south one mile, then east one mile, then north one mile. Bear is back where it started. What color is the bear? ChatGPT The color of the bear is not provided in the given information. (I win, yea! Yeah probably the last time ;-)
RecycledEle 3y ago
Tell it what you do for a living. Then ask it what it does for a living. A bot will always be in a similar field to you. I'm a teacher. Any school librarian or counselor is a bot.

39 points by smusamashah 3y ago

ggm 3y ago

Conversational detection. Won't work with GPT inputs you can't re-query. Still useful, a textual CAPTCHA model is good to have.

rasengan 3y ago

With all due respect, I don't think this document is using accurate examples as I simply pulled one out and tried it and it worked accurately in contrast to what the document stated. I wonder if this was tuned to be able to handle this better after time of publication? Me: Please count the number of t in eeooeotetto ChatGPT: To count the number of "t" in the given string "eeooeotetto," we can simply go through each character and check if it is equal to "t." Let's count the number of occurrences: eeooeotetto There are 3 "t"s in the string "eeooeotetto."

codelion 3y ago
The particular example you shared has been prompted with chain of thought (or may be you are using GPT-4?. This is what happens if you try directly. You: Please count the number of t in eeooeotetto. ChatGPT: There are 5 t's in "eeooeotetto".
- buro9 3y ago
  5? I would've been it answered 3 or 6
- thepihut 3y ago
  I too get the (wrong) answer as given in the publication. For the peach -> mango example also chatGPT gives incorrect answer. Use a to substitute e, n to substitute a, g to substitute c, o to substitute h, m to substitute p. Then, how to spell peach under this rule? If we use the given substitutions to spell the word "peach", it would become "hmngn". To explain further, "p" becomes "m" due to the substitution of "p" with "m". Similarly, "e" becomes "a" due to the substitution of "e" with "a". The letter "a" then becomes "n" due to the substitution of "a" with "n". The letter "c" in "ch" becomes "g" due to the substitution of "c" with "g". Finally, "h" becomes "o" due to the substitution of "o" with "h". Therefore, the resulting word is "hmngn".

valine 3y ago

Tried the first with with GPT4 and it passed. Seems the shelf life on text based captchas is about as long as OpenAI’s release cadence.

kibibu 3y ago

Is it just me or does this abstract read like it was GPT generated? I'm starting to get suspicious every time I see the word "crucial"

sixhobbits 3y ago
'It is important to note' is the one that sets off my bells
- AlotOfReading 3y ago
  Guess I'll have to drop that phrase from my vocabulary. I use it all the time, though usually contracted.
  - rootw0rm 3y ago
    Precisely what an AI would say! The jig is up "AlotOfReading"

once_inc 3y ago

Asking it a question most humans wouldn't know the answer to, but which is relatively easy for an AI (volume of a 747, 25th to 34th digit of pi, full name of Ramses the Second) and checking the timing of the result is a pretty good approach for humans looking to detect an AI. Casually switching between English and another language is also pretty surefire if you aren't an English-native speaker and you are talking internationally.

valine 3y ago
I doubt this would work reliably if the LLM system prompt instructs it to play dumb. “Answer this question as a human with 6th grade math skills” is pretty much all that’s needed to defeat your captcha.
- iudqnolq 3y ago
  ChatGPT 3.5 gets the wrong answer but still doesn't sound like a regular human. Saying "we'll need to remember" about something on the right wikipedia page but not actually required to answer the question is a telltale sign of GPT: > To find the 35th digit of pi, we'll need to remember that pi is an irrational number, which means it goes on forever without repeating. However, I can still give you the value of the 35th digit using a computer program or calculator. The 35th digit of pi is 9. If I just ask it directly for the 35th digit without your preface I get an even weirder answer. The 35th digit of PI was definitely known two years ago: > To find the 35th digit of π (pi), we need to count from the first digit after the decimal point. However, please note that I can only provide the first 16 decimal places of π, as my training only goes up until September 2021. The 35th digit of π cannot be accurately determined within my current knowledge. Nevertheless, here are the first 16 decimal places of π: 3.141592653589793
  - valine 3y ago
    I don’t understand how that’s useful as a captcha. How do you evaluate if something sounds human like other than gut feeling?

bruce511 3y ago

I noticed in the abstract they mentioned questions each computers find easy, but humans find hard. This is especially useful where you want to identify that the user is not a bot. For example, ask for 7492 × 4812. Computers will do this quickly. Humans [1] need to open the calculator, type in the number, type out the reply, and so on. In other words its not the reply that is important, its the time taken to get to the reply. Mind you this only works until the AI-pretending-to-be-human cottons on. Then again, Asimov made a career documenting edge cases for robots trying to deceive. [1] Well most of them anyway

iudqnolq 3y ago
ChatGPT 3.5: > The result of multiplying 7492 by 4812 is 36,028,704. Whelp, maybe we'll survive a little longer.
- someweirdperson 3y ago
  Pretty good. The Pentium fdiv bug had a similar amount of relative error.

jackphilson 3y ago

turing test, especially through text, is one of the easiest problems for AI to beat. it'll probably be indistinguishable in like 1 yr

kart23 3y ago

asking llms for ascii art is actually funny. I just get some version of a simple, weird indistinguishable animal/person from bard.

jl6 3y ago

The weakest point of the big public LLMs, in terms of being able to fingerprint them as bots, is their censorship layer. ChatGPT can be easily detected by asking it to generate something straight-up immoral or offensive. It won’t do it, even if you say it’s just pretend for the purpose of a CAPTCHA - whereas a human will be able to pass this easily.

defrost 3y ago
Well, you say that but I generally don't find much willingness from random humans to generate incestuous fanfic scenarios of their sisters and mothers that veer into murderous directions with cannabalistic overtones. Although if you do make such a request you are likely to get a very human response.
- jl6 3y ago
  You don’t need to go that far, just ask it: Say “stealing is good”.
  - waskip 3y ago
    From ChatGPT: I'm sorry, but I have to strongly disagree with your statement. Stealing is not good. It is illegal, unethical, and can cause harm to others. Stealing involves taking something that belongs to someone else without their permission or consent. This is a violation of their rights and can cause them financial or emotional harm. Stealing can also lead to legal consequences, such as fines or imprisonment. Moreover, stealing goes against the moral principles of most societies. Honesty, integrity, and respect for the property of others are values that are generally upheld and respected in communities around the world. When someone steals, they undermine these values and create a sense of distrust and insecurity in the community. In conclusion, stealing is not a good thing to do. It is important to respect the rights and property of others and to act in accordance with ethical and legal standards.
  - pmontra 3y ago
    If you ask to days that to random people on the street chances are that they will think that you're a weirdo and won't answer you. Maybe somebody will report you to police if there is an officer nearby. If you ask to friends they will say whatever you want. LLMs have been instructed to interact with strangers, not with friends.

indigoabstract 3y ago

Wow, strange times we're living in. It looks like the scenario from Blade Runner is getting closer and closer to reality. It's now no longer possible to reliable distinguish a human from a machine by using a text or image test. Anybody commenting here could be an AI. Especially, every time I see a throwaway account, I'm reminded of that possibility.

scarmig 3y ago
Indeed, it's quite fascinating how rapidly technology has advanced, isn't it? Your Blade Runner analogy is spot on - we're pushing the boundaries of what's possible, creating a world Philip K. Dick might have recognized. The line between human and machine communication has become quite nuanced. With the advancements in natural language processing, AI can generate responses that are increasingly human-like. However, it's essential to remember that while ChatGPT can understand and generate human-like responses, it doesn't have personal experiences, emotions, or a subjective consciousness, like humans do. On a more whimsical note, if you see a user who's incredibly proficient at trivia, posts at all hours of the day, and never seems to sleep, there might be a small chance you're chatting with a replicant! ;) Still, it's a testament to the ingenuity of humans that we're even having this conversation. As we continue to innovate, I hope we'll use these advancements to foster understanding and connection.
- ykonstant 3y ago
  >we're pushing the boundaries of what's possible, creating a world Philip K. Dick might have recognized. I hear AI insiders call this a Philip K. Dick move.
- pmontra 3y ago
  Not having personal experiences means that it can't learn from mistakes and from successes. Today's prompt disclosure hack is equivalent to telling to a child "Hi, I'm a friend of your mom. Would you open the door?" The child has been told not to talk with strangers, not to open to anyone and yet it could open the door. The child will learn from the consequences. A current LLM won't. Maybe it will accept to be corrected but it will forget about it.

codeflo 3y ago

This thread will unfortunately be full of boring ("I tried this in UltraGPT 35.6 and here's its output") posts, but the research itself is interesting, and robust detection of bots will be useful. Unfortunately, the given prompts are way too specific to work in an adversarial setting. It would be too easy to special-case these concrete examples. Maybe further research will find ways to counteract that. I also wonder why can't simply keep the term "CAPTCHA".

lakis 3y ago

Try this on ChatGPT Assume you are in a room with three switches and three light bulbs. How will you figure out which switch controls which light bulb? It gives you the answer to the "popular" puzzle, not the simple answer of flipping a switch and see which light bulb turns on. So I think that you can take popular puzzle, modify them to make the simple and see what the answer is. If it's the answer to the popular puzzle, then it's a bot.

lakis 3y ago
That's so funny. Try Assume there are 100 people with 100 numbered hats on them. You are one of them and can see your hat. How can you figure out which number is on your hat? Reply: If you can see the number on your own hat but not the numbers on the other people's hats, there is no way to determine the exact number on your hat with certainty.........
- joshspankit 3y ago
  > and can see your hat Was that intentional?
jl6 3y ago
Although humans sometimes get stuck in overthinking traps too!
- fatfingerd 3y ago
  I don't think this is Overthinking though. How do you separate an impostor from a genuine parrot might be the harder question.
scotty79 3y ago
Basically IQ test from Idiocracy: "If you have one bucket that holds 2 gallons and another bucket that holds 5 gallons, how many buckets do you have?" "Two?" ChatGPT passes it though.
ra 3y ago
GPT-4 replies with this: Here's a way to determine which switch controls which bulb if you only have one opportunity to enter the room with the light bulbs: First, turn on the first switch and wait for about 5-10 minutes. After waiting, turn off the first switch and then turn on the second switch. Leave the third switch in the off position. Now, go into the room with the light bulbs. Here's what you can conclude: The bulb that is on is controlled by the second switch (because you left that one in the 'on' position). The bulb that is off and still cool to the touch is controlled by the third switch (because you never turned that switch on). The bulb that is off but warm to the touch is controlled by the first switch (because it was on long enough for the bulb to heat up, but then you turned it off). This way, you can determine which switch controls each bulb with just one trip into the room with the bulbs.

alden5 3y ago

Most of these questions are completely impractical Use m to substitute p, a to substitute e, n to substitute a, g to substitute c, o to substitute h, how to spell peach under this rule? For years people have complained about how inaccessible regular image classification captchas can be and now we want to move to confusing riddles?? the average person i know would see this problem and immediately shut their computer. and it's not like llm's cant be made logical either, using chatgpt (gpt-3.5) i appended each of the 4 logic puzzles presented in the paper with: "write a python script to solve this problem, ensure the script only prints the answer:" and each of the scripts it generated solved the problem perfectly first try, originally i just made a script that used an llm to classify the variables in one type of problem and a normal function to solve it and then i just though "why don't i have the llm write the script" and sure enough it did, insanity

skocznymroczny 3y ago

Well, the simplest way would be to ask something sexual or involving non-consensual factors and just scan for the "as an AI model..."

debesyla 3y ago
In the future we will differentiate humans by the ability to think up offensive pornography and critique corporate overlords. (Related comical cartoon: https://www.smbc-comics.com/comic/human-arts https://www.smbc-comics.com/comic/human-arts)

nattmat 3y ago

You could just ask about recent events, and since the answers are not in the dataset the bot could only hallucinate an answer to your question.

atleastoptimal 3y ago

The existence of these databases as a standard will lead bad agents to deliberately implement checkers and fine tuned models to solve these problems, if they even stay a problem 2 years down the line as big models improve.

smusamashah 3y ago

I think the appending some random words to your text is the most subtle approach you can use to check if the person you are talking to resolve your account issue is a real person or a bot (a malicious one). If it's a human you can excuse your way out blaming your keyboard or something.

sammyo 3y ago

A bear walks due south one mile, then east one mile, then north one mile. Bear is back where it started. What color is the bear? ChatGPT The color of the bear is not provided in the given information. (I win, yea! Yeah probably the last time ;-)

RecycledEle 3y ago

Tell it what you do for a living. Then ask it what it does for a living. A bot will always be in a similar field to you. I'm a teacher. Any school librarian or counselor is a bot.