4 ms·

It can appear reasonably smart on the surface, but all it is is a stochastic parrot. It cannot reason with you about the code. To best illustrate what I mean,

It can appear reasonably smart on the surface, but all it is is a stochastic parrot. It cannot reason with you about the code. To best illustrate what I mean, watch this chess match[0] it's quite riveting. Since it read millions of matches, it can predict a legal move most of the time, and even some good moves some of the time, but it cannot "understand" the rules of chess, and makes some hilariously illegal moves, especially if the match lasts longer. [0] https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_placed_stockfish_white_against_chatgpt_black/ https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_pla...

Vanclief 4y ago
Exactly and I personally think that will always be the largest limiter to how good can the technology get. No matter how good the stochastic parrot gets, its still a parrot.
Verdex 4y ago
On a similar vein, I tried to get chatgpt to play wordle. The result looked something like: Me: crane GPT: _ _ _ _ e Me: moist GPT: _ _ _ r _ Me: glyph GPT: you guessed it, the word was glyph Now, maybe GPT 4 or other future developments will give better results, but to me this highlights exactly what you're saying. LLMs do not have an internal structure in their 'minds' that they're pondering about. It's a very impressive engine for guessing the next character to produce into a stream. There's definitely usages for this, but not what a lot of people are saying.
- mtrycz2 4y ago
  > you guessed it, the word was glyph My pet conspiracy theory is that is is wired to please the user, to get better coverage from the media and social media.
  - codetrotter 4y ago
    I don’t think so. In Wordle you have to guess the word in six attempts. It’s a fun game and often simple. So it could be that ChatGPT picked up on a pattern in the training data where after a couple of guesses, a lot of the time people pick the right word. So statistically it might go like. Guess a word. Probably not the right one. Guess a couple more and suddenly it’s statistically likely to be the correct word, and because of that the LLM ends up outputting the congrats and so on
  - tome 4y ago
    I think there's something to this. I have a theory that LLMs are implicitly trained primarily to impress people, since that's what motivates those who work on them, excites the general public, and convinces conferences to publish papers.
    - evnc 4y ago
      In a sense, this is exactly what RLHF is, right?
      - tome 4y ago
        I'm thinking of something at a larger scale. In some sense models that "wow" society get more interest and funding.
- skybrian 4y ago
  This an interesting example because, due to the API being stateless, it cannot have any solution “in mind.” It’s going to reply as if someone else started the game and it’s taking their place for the current turn. Even if it reasoned ideally, it has to check if your answer is consistent, and if it is, decide whether to let you win or to find another solution. If your answer is inconsistent, it effectively has to play wordle by thinking of another answer. It doesn’t seem all that surprising that it didn’t figure it out.
  - cocoflunchy 4y ago
    I tried to get it to remember its word by asking it to give it to me in b64: (GPT-4) > We will play the word game wordle. Start by picking a 5 letter word. Display it in b64 so I can check later that you kept the same word all along. I will submit my guesses and you will show the letters that are correct, we will keep going until I have found the word > Great, I've picked a 5-letter word and encoded it in base64. Here's the encoded word: Q2l0eWw=. You can now start submitting your guesses! But after a few tries it was clear it was not working so I gave up: > I give up, what's the word? > Before I reveal the word, let me first decode the base64 string I provided earlier to ensure that I'm giving you the correct word. The base64 string was: Q2l0eWw=. After decoding the base64 string, the word I chose is "citly". However, I apologize for the confusion caused during the game, as "citly" is not a valid English word. I made an error while choosing the word. Please accept my apologies for any inconvenience caused during the game. However asking it for a python program to play the game worked perfectly the first time, try it here: https://replit.com/@CorentinSmith1/gpt-wordle#main.py https://replit.com/@CorentinSmith1/gpt-wordle#main.py
    - Verdex 4y ago
      [As an aside, is the GPT feedback functionality correct? For example, if the secret word is "peals" and the user's guess is "apple" the feedback is going to be "a?p?p?l+e?". However, I would expect the feedback to be "a?p?_l+e?". Because the first 'p' exists in the secret but is in the wrong place and the second 'p' does not exist in the secret word.]
- shagie 4y ago
  As a slight correction, it isn't next character but rather next token. https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-... > Tokens can be thought of as pieces of words. Before the API processes the prompts, the input is broken down into tokens. These tokens are not cut up exactly where the words start or end - tokens can include trailing spaces and even sub-words. Here are some helpful rules of thumb for understanding tokens in terms of lengths: > ... > Wayne Gretzky’s quote "You miss 100% of the shots you don't take" contains 11 tokens. https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer It isn't going character by character, but rather token by token - both for input and for output. This also helps explain why it has trouble with breaking a word apart (as in the case of wordle) because it doesn't "think" of glyph as 5 letters but rather two tokens that happen to be 'gly' and 'ph' with the ids of [10853, 746].
hirundo 4y ago
It is humbling, if not humiliating, that a stochastic parrot can reproduce such a significant chunk of human intelligence. The association elevates stochastic parrots more than it denigrates LLMs.
seeEllArr 4y ago
[dead]
medvezhenok 4y ago
(1) GPT-4 already had a large improvement over ChatGPT (2) Changing the prompting reduced the illegal moves to almost 0 (3) There have been experiments that show GPT has a internal "state" of the world and can do simple reasoning puzzles. This model of the world evolves with each generation. I understand the skepticism, but don't let that blind you to the reality of the technology. I'm a skeptic at heart, and I could immediately tell GPT was a game-changer. It can already replace half of the ML models that are used at my job and do it better (if it was economical enough).

Vanclief 4y ago

Exactly and I personally think that will always be the largest limiter to how good can the technology get. No matter how good the stochastic parrot gets, its still a parrot.

Verdex 4y ago

On a similar vein, I tried to get chatgpt to play wordle. The result looked something like: Me: crane GPT: _ _ _ _ e Me: moist GPT: _ _ _ r _ Me: glyph GPT: you guessed it, the word was glyph Now, maybe GPT 4 or other future developments will give better results, but to me this highlights exactly what you're saying. LLMs do not have an internal structure in their 'minds' that they're pondering about. It's a very impressive engine for guessing the next character to produce into a stream. There's definitely usages for this, but not what a lot of people are saying.

mtrycz2 4y ago
> you guessed it, the word was glyph My pet conspiracy theory is that is is wired to please the user, to get better coverage from the media and social media.
- codetrotter 4y ago
  I don’t think so. In Wordle you have to guess the word in six attempts. It’s a fun game and often simple. So it could be that ChatGPT picked up on a pattern in the training data where after a couple of guesses, a lot of the time people pick the right word. So statistically it might go like. Guess a word. Probably not the right one. Guess a couple more and suddenly it’s statistically likely to be the correct word, and because of that the LLM ends up outputting the congrats and so on
- tome 4y ago
  I think there's something to this. I have a theory that LLMs are implicitly trained primarily to impress people, since that's what motivates those who work on them, excites the general public, and convinces conferences to publish papers.
  - evnc 4y ago
    In a sense, this is exactly what RLHF is, right?
    - tome 4y ago
      I'm thinking of something at a larger scale. In some sense models that "wow" society get more interest and funding.
skybrian 4y ago
This an interesting example because, due to the API being stateless, it cannot have any solution “in mind.” It’s going to reply as if someone else started the game and it’s taking their place for the current turn. Even if it reasoned ideally, it has to check if your answer is consistent, and if it is, decide whether to let you win or to find another solution. If your answer is inconsistent, it effectively has to play wordle by thinking of another answer. It doesn’t seem all that surprising that it didn’t figure it out.
- cocoflunchy 4y ago
  I tried to get it to remember its word by asking it to give it to me in b64: (GPT-4) > We will play the word game wordle. Start by picking a 5 letter word. Display it in b64 so I can check later that you kept the same word all along. I will submit my guesses and you will show the letters that are correct, we will keep going until I have found the word > Great, I've picked a 5-letter word and encoded it in base64. Here's the encoded word: Q2l0eWw=. You can now start submitting your guesses! But after a few tries it was clear it was not working so I gave up: > I give up, what's the word? > Before I reveal the word, let me first decode the base64 string I provided earlier to ensure that I'm giving you the correct word. The base64 string was: Q2l0eWw=. After decoding the base64 string, the word I chose is "citly". However, I apologize for the confusion caused during the game, as "citly" is not a valid English word. I made an error while choosing the word. Please accept my apologies for any inconvenience caused during the game. However asking it for a python program to play the game worked perfectly the first time, try it here: https://replit.com/@CorentinSmith1/gpt-wordle#main.py https://replit.com/@CorentinSmith1/gpt-wordle#main.py
  - Verdex 4y ago
    [As an aside, is the GPT feedback functionality correct? For example, if the secret word is "peals" and the user's guess is "apple" the feedback is going to be "a?p?p?l+e?". However, I would expect the feedback to be "a?p?_l+e?". Because the first 'p' exists in the secret but is in the wrong place and the second 'p' does not exist in the secret word.]
shagie 4y ago
As a slight correction, it isn't next character but rather next token. https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-... > Tokens can be thought of as pieces of words. Before the API processes the prompts, the input is broken down into tokens. These tokens are not cut up exactly where the words start or end - tokens can include trailing spaces and even sub-words. Here are some helpful rules of thumb for understanding tokens in terms of lengths: > ... > Wayne Gretzky’s quote "You miss 100% of the shots you don't take" contains 11 tokens. https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer It isn't going character by character, but rather token by token - both for input and for output. This also helps explain why it has trouble with breaking a word apart (as in the case of wordle) because it doesn't "think" of glyph as 5 letters but rather two tokens that happen to be 'gly' and 'ph' with the ids of [10853, 746].

hirundo 4y ago

It is humbling, if not humiliating, that a stochastic parrot can reproduce such a significant chunk of human intelligence. The association elevates stochastic parrots more than it denigrates LLMs.

seeEllArr 4y ago

[dead]

medvezhenok 4y ago

(1) GPT-4 already had a large improvement over ChatGPT (2) Changing the prompting reduced the illegal moves to almost 0 (3) There have been experiments that show GPT has a internal "state" of the world and can do simple reasoning puzzles. This model of the world evolves with each generation. I understand the skepticism, but don't let that blind you to the reality of the technology. I'm a skeptic at heart, and I could immediately tell GPT was a game-changer. It can already replace half of the ML models that are used at my job and do it better (if it was economical enough).