6 ms·

Wow, they asked the model to self-evaluate and it just outright cheated: He has three cats. Proposed: h’io’ngkiltrikumrikumrikumri’nguuy Correct: h

Wow, they asked the model to self-evaluate and it just outright cheated: He has three cats. Proposed: h’io’ngkiltrikumrikumrikumri’nguuy Correct: h’io’ngkiltri’ngkumrikumri’nguuy Points: 1 Hypothesis: N/A (Other comments observe that it accidentally compensated for this by getting the sum wrong, haha, d'oh) I have had similar problems with trying to get ChatGPT to do nontrivial things, "here are the rules for this game, do you understand this game, great, let's play it." And then it's like herding cats. "No that's wrong, the game pieces cannot leave the game board," "Oh my apologies you are entirely correct, here is the revised board (proceeds to dump the exact same state of the game board that I told it was wrong)." Eventually it will lie about its own capacities, "As an AI language model I am incapable of selecting a move to play next"... But you have done several already!!! This is literally the ONLY thing you have been doing right and now you refuse? Some other prompts are more successful but it does seem to have a sing-song high school book review style that inclines it to be boring... Very uncanny valley.

sharkweek 4y ago
I was trying for 20 minutes to get it to spit out all 50 state capitals with the city names in alphabetical order and it kept doing two things: 1) It'd put the list in alphabetical order by state, but it'd include all the correct capitals 2) It'd list 49 of the 50 capitals, in alphabetical order this time, but duplicating Madison, WI. I'd ask it to try and figure out what it did wrong in both cases, and it'd correctly identify the mistake, but then repeat it. Not sure how I got there eventually, but on about the 7th or 8th attempt, it got it right.
- Daneel_ 4y ago
  Not here to one-up you, but currently this is just down to how you ask. I came up with this in about a minute: "Please list all 50 US state capital cities, with the list sorted alphabetically starting at the first letter of each line of your response. Please do not create sections for each letter." This returned: - Albany, New York - Annapolis, Maryland - Atlanta, Georgia - Augusta, Maine - Austin, Texas - Baton Rouge, Louisiana - Bismarck, North Dakota - ... My gut feeling is that to get what you want from it you need to have a solid understanding of how to manipulate search engines and other fuzzy input systems. On self reflection I find it interesting that I wrote "Please" at the start of each sentence, as if that would give me a better output. Heh.
  - melagonster 4y ago
    I sure I read this somewhere, more please and thank you offer better output. the problem is, putting all requirements in one sentence is a rough behaviour, but chatGPT work better in this condition. I always forget this:/
  - sharkweek 4y ago
    Maybe, but I also think it’s fair to assume when you ask it in a much simpler way, “please list all US state capitals in alphabetical order” it should be able to do so. That isn’t a complicated question that requires much refinement to understand. To your “please” comment, I also find myself using common niceties probably out of social habit, but also perhaps because in the back of my mind I want the AI that inevitably takes over to remember me being nice to it (half-joking, but also kind of not).
    - gwright 4y ago
      I feel like your observation that this "isn't a complicated question" is leaning on an implicit assumption that ChatGPT is a general AI and not a LLM. It is just generating text based on probabilities -- it isn't "reasoning". I might go as far to say that inferences computed by a LLM are of all the same complexity but I don't really know enough about ChatGPT to be confident in that statement.
      - nearbuy 4y ago
        People keep repeating that LLMs are "just generating text based on probabilities". That statement doesn't mean anything. I think people who say this are imagining LLMs work something like a statistical model. Maybe it's doing a linear regression or works like a Markov chain. It's not. A single artificial neuron sort of works like that. But that's sort of like saying a single transistor is just an electronically controlled switch, so the only thing computers can do is switching. It's true in some sense that computers are just doing a lot of switching, but it turns out all this switching is Turing-complete. That means computers can theoretically compute anything that's possible to compute given enough time and memory, which includes anything a human could figure out. Similar principle applies to LLMs. Using probabilities is part of what they do, but that doesn't preclude them from using logic and rules of inference.
- kingkongjaffa 4y ago
  Another thing you can try which is readily testable is Pokémon. So in the Pokémon games, different Pokémon have different stats, for example attack, defence, speed, and so on. I would ask gpt to list in a table pokemon with more than a specific amount in a stat like >150 attack for example. It routinely can print Pokémon in a table that matches the criteria, but it misses some out or includes wrong ones. When you point it out it agrees with you and will try to reprint the table but each time there are some errors, missing Pokémon, Pokémon incorrectly added. The dataset is very well known so it’s a good benchmark of correctness. Example: https://pokemondb.net/pokedex/all https://pokemondb.net/pokedex/all This was in gpt3.5
- nearbuy 4y ago
  Being able to sort a large list like that is really impressive given how ChatGPT works. I wonder if it's seen enough sorted lists of capitals in its training data to help it. ChatGPT doesn't have any internal working memory. It can't think things out in its head before forming an answer. It formulates its answer one token at a time, but it only gets one pass through its network to make each token. (One token is often about 4 characters, but for simplicity I'll pretend it generates one word at a time.) To sort a list of 50 things, it has to figure out in a single pass which thing comes first. Then in the next pass, which thing comes second. And so on. There's no internal memory between passes either; it only sees what's written in the chat (including its own writing). It can't do loops or iteration internally within a pass either. A human could do this iteratively for any number of items. You just go through the items one by one, keeping track of the earliest alphabetical item so far (using your working memory). Once you've gone through the entire list and found the first item, cross that one off and repeat the process until the entire list is sorted. ChatGPT can't do it that way.
- int_19h 4y ago
  It doesn't help with repetition, but when it comes to force structure on the output data, this approach looks interesting: https://github.com/newhouseb/clownfish https://github.com/newhouseb/clownfish TL;DR: it exploits the fact that the model returns probabilities for all the possible following tokens to enforce a JSON schema on the output as it is produced, backtracking as needed.
adventured 4y ago
There is some amusement in that if we manage to build various AGI, these aspects will tend to get drastically worse rather than better. They'll be exceptional at fraud, cheating, lying, stealing, manipulation. Far beyond anything humans have ever managed. People think fake news is a problem today, they haven't seen anything yet. The Europeans will move on it first and attempt to hyper regulate it to stop it or slow it down. They'll fail.
jacobsenscott 4y ago
Chat gpt can't do these things because it doesn't know it is doing anything with a goal. It doesn't know it is playing a game for example. It doesn't know what a game is.
- roflyear 4y ago
  Correct. How do so many people get this wrong?
  - ratg13 4y ago
    Movies, comics, books, sci-fi Everyone has been primed to accept AI as an intelligent companion, not a talking search engine.
- joshribakoff 4y ago
  The counter argument is if you just ask it, it will readily explain what a game is. How do we know you (as a human) actually know what a game is? I would have to test your knowledge and/or take your word for it.
  - z3c0 4y ago
    Being able to discern its word soup as a description of what constitutes a game is more a demonstration of you understanding what a game is, not it. And therein lies the magic of ChatGPT: it knows nothing, but the projected significance of the token arrangements it creates tugs on what we know, and that causes a profound feeling akin to interacting with something intelligent.
    - joshribakoff 4y ago
      Lets rephrase this differently, no one can prove whether it “understands” or not, and the same for humans. No one can prove whether consciousness is an “illusion” for humans. Your brain might just use embeddings, too. If you ask an average person if children understand what a game is, “yes” would be a common answer, but just like an AI model it may not have enough focus to play the game properly. I believe this is the whole point behind “the turing test”. If you instead argued it can’t experience emotion, i would agree.
leereeves 4y ago
> "No that's wrong, the game pieces cannot leave the game board," Even with approximately all the training data on Earth, ChatGPT still does that with chess. https://www.youtube.com/watch?v=GneReITaRvs https://www.youtube.com/watch?v=GneReITaRvs
jackmott 4y ago
[dead]
curiousgal 4y ago
In the context of chatgpt words like "cheated", "compensated", "lied" and any that imply actual intention genuinely make me cringe.

sharkweek 4y ago

I was trying for 20 minutes to get it to spit out all 50 state capitals with the city names in alphabetical order and it kept doing two things: 1) It'd put the list in alphabetical order by state, but it'd include all the correct capitals 2) It'd list 49 of the 50 capitals, in alphabetical order this time, but duplicating Madison, WI. I'd ask it to try and figure out what it did wrong in both cases, and it'd correctly identify the mistake, but then repeat it. Not sure how I got there eventually, but on about the 7th or 8th attempt, it got it right.

Daneel_ 4y ago
Not here to one-up you, but currently this is just down to how you ask. I came up with this in about a minute: "Please list all 50 US state capital cities, with the list sorted alphabetically starting at the first letter of each line of your response. Please do not create sections for each letter." This returned: - Albany, New York - Annapolis, Maryland - Atlanta, Georgia - Augusta, Maine - Austin, Texas - Baton Rouge, Louisiana - Bismarck, North Dakota - ... My gut feeling is that to get what you want from it you need to have a solid understanding of how to manipulate search engines and other fuzzy input systems. On self reflection I find it interesting that I wrote "Please" at the start of each sentence, as if that would give me a better output. Heh.
- melagonster 4y ago
  I sure I read this somewhere, more please and thank you offer better output. the problem is, putting all requirements in one sentence is a rough behaviour, but chatGPT work better in this condition. I always forget this:/
- sharkweek 4y ago
  Maybe, but I also think it’s fair to assume when you ask it in a much simpler way, “please list all US state capitals in alphabetical order” it should be able to do so. That isn’t a complicated question that requires much refinement to understand. To your “please” comment, I also find myself using common niceties probably out of social habit, but also perhaps because in the back of my mind I want the AI that inevitably takes over to remember me being nice to it (half-joking, but also kind of not).
  - gwright 4y ago
    I feel like your observation that this "isn't a complicated question" is leaning on an implicit assumption that ChatGPT is a general AI and not a LLM. It is just generating text based on probabilities -- it isn't "reasoning". I might go as far to say that inferences computed by a LLM are of all the same complexity but I don't really know enough about ChatGPT to be confident in that statement.
    - nearbuy 4y ago
      People keep repeating that LLMs are "just generating text based on probabilities". That statement doesn't mean anything. I think people who say this are imagining LLMs work something like a statistical model. Maybe it's doing a linear regression or works like a Markov chain. It's not. A single artificial neuron sort of works like that. But that's sort of like saying a single transistor is just an electronically controlled switch, so the only thing computers can do is switching. It's true in some sense that computers are just doing a lot of switching, but it turns out all this switching is Turing-complete. That means computers can theoretically compute anything that's possible to compute given enough time and memory, which includes anything a human could figure out. Similar principle applies to LLMs. Using probabilities is part of what they do, but that doesn't preclude them from using logic and rules of inference.
kingkongjaffa 4y ago
Another thing you can try which is readily testable is Pokémon. So in the Pokémon games, different Pokémon have different stats, for example attack, defence, speed, and so on. I would ask gpt to list in a table pokemon with more than a specific amount in a stat like >150 attack for example. It routinely can print Pokémon in a table that matches the criteria, but it misses some out or includes wrong ones. When you point it out it agrees with you and will try to reprint the table but each time there are some errors, missing Pokémon, Pokémon incorrectly added. The dataset is very well known so it’s a good benchmark of correctness. Example: https://pokemondb.net/pokedex/all https://pokemondb.net/pokedex/all This was in gpt3.5
nearbuy 4y ago
Being able to sort a large list like that is really impressive given how ChatGPT works. I wonder if it's seen enough sorted lists of capitals in its training data to help it. ChatGPT doesn't have any internal working memory. It can't think things out in its head before forming an answer. It formulates its answer one token at a time, but it only gets one pass through its network to make each token. (One token is often about 4 characters, but for simplicity I'll pretend it generates one word at a time.) To sort a list of 50 things, it has to figure out in a single pass which thing comes first. Then in the next pass, which thing comes second. And so on. There's no internal memory between passes either; it only sees what's written in the chat (including its own writing). It can't do loops or iteration internally within a pass either. A human could do this iteratively for any number of items. You just go through the items one by one, keeping track of the earliest alphabetical item so far (using your working memory). Once you've gone through the entire list and found the first item, cross that one off and repeat the process until the entire list is sorted. ChatGPT can't do it that way.
int_19h 4y ago
It doesn't help with repetition, but when it comes to force structure on the output data, this approach looks interesting: https://github.com/newhouseb/clownfish https://github.com/newhouseb/clownfish TL;DR: it exploits the fact that the model returns probabilities for all the possible following tokens to enforce a JSON schema on the output as it is produced, backtracking as needed.

adventured 4y ago

There is some amusement in that if we manage to build various AGI, these aspects will tend to get drastically worse rather than better. They'll be exceptional at fraud, cheating, lying, stealing, manipulation. Far beyond anything humans have ever managed. People think fake news is a problem today, they haven't seen anything yet. The Europeans will move on it first and attempt to hyper regulate it to stop it or slow it down. They'll fail.

jacobsenscott 4y ago

Chat gpt can't do these things because it doesn't know it is doing anything with a goal. It doesn't know it is playing a game for example. It doesn't know what a game is.

roflyear 4y ago
Correct. How do so many people get this wrong?
- ratg13 4y ago
  Movies, comics, books, sci-fi Everyone has been primed to accept AI as an intelligent companion, not a talking search engine.
joshribakoff 4y ago
The counter argument is if you just ask it, it will readily explain what a game is. How do we know you (as a human) actually know what a game is? I would have to test your knowledge and/or take your word for it.
- z3c0 4y ago
  Being able to discern its word soup as a description of what constitutes a game is more a demonstration of you understanding what a game is, not it. And therein lies the magic of ChatGPT: it knows nothing, but the projected significance of the token arrangements it creates tugs on what we know, and that causes a profound feeling akin to interacting with something intelligent.
  - joshribakoff 4y ago
    Lets rephrase this differently, no one can prove whether it “understands” or not, and the same for humans. No one can prove whether consciousness is an “illusion” for humans. Your brain might just use embeddings, too. If you ask an average person if children understand what a game is, “yes” would be a common answer, but just like an AI model it may not have enough focus to play the game properly. I believe this is the whole point behind “the turing test”. If you instead argued it can’t experience emotion, i would agree.

leereeves 4y ago

> "No that's wrong, the game pieces cannot leave the game board," Even with approximately all the training data on Earth, ChatGPT still does that with chess. https://www.youtube.com/watch?v=GneReITaRvs https://www.youtube.com/watch?v=GneReITaRvs

jackmott 4y ago

[dead]

curiousgal 4y ago

In the context of chatgpt words like "cheated", "compensated", "lied" and any that imply actual intention genuinely make me cringe.