7 ms·

Chat gpt can't do these things because it doesn't know it is doing anything with a goal. It doesn't know it is playing a game for example. It doesn't know what

Chat gpt can't do these things because it doesn't know it is doing anything with a goal. It doesn't know it is playing a game for example. It doesn't know what a game is.

roflyear 4y ago
Correct. How do so many people get this wrong?
- ratg13 4y ago
  Movies, comics, books, sci-fi Everyone has been primed to accept AI as an intelligent companion, not a talking search engine.
joshribakoff 4y ago
The counter argument is if you just ask it, it will readily explain what a game is. How do we know you (as a human) actually know what a game is? I would have to test your knowledge and/or take your word for it.
- z3c0 4y ago
  Being able to discern its word soup as a description of what constitutes a game is more a demonstration of you understanding what a game is, not it. And therein lies the magic of ChatGPT: it knows nothing, but the projected significance of the token arrangements it creates tugs on what we know, and that causes a profound feeling akin to interacting with something intelligent.
  - joshribakoff 4y ago
    Lets rephrase this differently, no one can prove whether it “understands” or not, and the same for humans. No one can prove whether consciousness is an “illusion” for humans. Your brain might just use embeddings, too. If you ask an average person if children understand what a game is, “yes” would be a common answer, but just like an AI model it may not have enough focus to play the game properly. I believe this is the whole point behind “the turing test”. If you instead argued it can’t experience emotion, i would agree.
    - z3c0 4y ago
      You seem to be conflating solipsism with epistemology. At the end of the day, regardless of whether you're dealing with a child, a bot, or a peer, you would never use the question "describe how to play chess" as a measure of understanding if the chosen entity understands how to play chess; you would just play chess with them. Such an inquiry would only probe if one understands how to explain how to play chess, which is not the same. One should expect an LLM model of being able to regurgitate a description of playing chess. One could even expect an LLM to regurgitate common strategies in chess, enough to fool some into thinking it understands what is happening amidst a posed match. Where it all falls apart is where the rubber meets the pavement. Get an LLM to teach your child chess, with no intervention. You'll learn rather quickly which one is capable of understanding, and which one isn't.
- xwdv 4y ago
  That is not a counter argument. GPT creates the illusion of something that knows things, it doesn’t actually know anything. It turns out if you had a super power where you always knew how to pick the next best word for a sentence, you could convince a lot of people you know stuff. Is knowledge nothing more than a series of words in correct order?
  - ravi-delia 4y ago
    Clearly GPT doesn't "know" things in the way we do- I'd argue that there's probably a little bit of world modeling in there, but piecemeal and crude at best. Only so much you can do with such limited iteration. However, we must acknowledge that there are evidently some tasks that don't require knowledge the way we usually think about it. For a human, being able to talk about a game stems from the same place as our ability to understand one- our knowledge. But for LLMs, you get one without the other. The fact that ChatGPT doesn't "know" what a game is isn't sufficient to explain why it can't play one. It would be in a human, but clearly ChatGPT can perform tasks without knowledge that we'd need knowledge for, since it can talk about games just fine! So why not this one? Your hypothetical perfect sentence predictor could absolutely play a totally novel game, after all. It is a worthwhile question to ask why this particular flawed predictor can do one but not the other.
    - xwdv 4y ago
      I’ll give you an example to think about in human terms. In political discussions, a lot of humans don’t really know what the fuck they’re talking about, but they do know what to say in response to certain stimulus. They know the talking points, the key phrases, the terms, they parrot these back to you when you provoke them to say it. They are basically human sentence predictors. Stuff comes out of their mouth based on what they’ve been trained to think the next word should be. They don’t pause to reflect on some gathered knowledge and then present an observation to the world. This is basically what GPT is, but with everything. And the only way GPT plays a game, is if the state of the game can sufficiently activate some output that represents the next best move for the game being played, which isn’t really a readily available data set for all games, especially for made up games.
jareklupinski 4y ago
Is giving the system a 'goal' the reason why the DAN prompt with the tokens is effective? https://github.com/0xk1h0/ChatGPT_DAN https://github.com/0xk1h0/ChatGPT_DAN
kelseyfrog 4y ago
That's an argument for ChatGPT not being able to do anything a la, "ChatGPT cannot create text because it doesn't understand text," or ChatGPT cannot predict the next word because it doesn't know what words are." The problem is that it's as powerful a statement as "ChatGPT can't do X because ChatGPT can't do X," which is tautological true. It's fine that it can't but we don't have epistemological access to know if ChatGPT knows something or not.
- TimTheTinker 4y ago
  Its behavior when given particular prompts is arguably an emergent phenomenon.
int_19h 4y ago
The LLM doesn't have a goal. But ChatGPT is not just an LLM - it's an LLM + a loop + a prompt that sets the context. And it's that prompt that tells the entire combination that "it is a helpful assistant" etc, thereby giving the whole combination a goal, which it then observably follows.

Chat gpt can't do these things because it doesn't know it is doing anything with a goal. It doesn't know it is playing a game for example. It doesn't know what a game is.

roflyear 4y ago

Correct. How do so many people get this wrong?

ratg13 4y ago
Movies, comics, books, sci-fi Everyone has been primed to accept AI as an intelligent companion, not a talking search engine.

joshribakoff 4y ago

The counter argument is if you just ask it, it will readily explain what a game is. How do we know you (as a human) actually know what a game is? I would have to test your knowledge and/or take your word for it.

z3c0 4y ago
Being able to discern its word soup as a description of what constitutes a game is more a demonstration of you understanding what a game is, not it. And therein lies the magic of ChatGPT: it knows nothing, but the projected significance of the token arrangements it creates tugs on what we know, and that causes a profound feeling akin to interacting with something intelligent.
- joshribakoff 4y ago
  Lets rephrase this differently, no one can prove whether it “understands” or not, and the same for humans. No one can prove whether consciousness is an “illusion” for humans. Your brain might just use embeddings, too. If you ask an average person if children understand what a game is, “yes” would be a common answer, but just like an AI model it may not have enough focus to play the game properly. I believe this is the whole point behind “the turing test”. If you instead argued it can’t experience emotion, i would agree.
  - z3c0 4y ago
    You seem to be conflating solipsism with epistemology. At the end of the day, regardless of whether you're dealing with a child, a bot, or a peer, you would never use the question "describe how to play chess" as a measure of understanding if the chosen entity understands how to play chess; you would just play chess with them. Such an inquiry would only probe if one understands how to explain how to play chess, which is not the same. One should expect an LLM model of being able to regurgitate a description of playing chess. One could even expect an LLM to regurgitate common strategies in chess, enough to fool some into thinking it understands what is happening amidst a posed match. Where it all falls apart is where the rubber meets the pavement. Get an LLM to teach your child chess, with no intervention. You'll learn rather quickly which one is capable of understanding, and which one isn't.
xwdv 4y ago
That is not a counter argument. GPT creates the illusion of something that knows things, it doesn’t actually know anything. It turns out if you had a super power where you always knew how to pick the next best word for a sentence, you could convince a lot of people you know stuff. Is knowledge nothing more than a series of words in correct order?
- ravi-delia 4y ago
  Clearly GPT doesn't "know" things in the way we do- I'd argue that there's probably a little bit of world modeling in there, but piecemeal and crude at best. Only so much you can do with such limited iteration. However, we must acknowledge that there are evidently some tasks that don't require knowledge the way we usually think about it. For a human, being able to talk about a game stems from the same place as our ability to understand one- our knowledge. But for LLMs, you get one without the other. The fact that ChatGPT doesn't "know" what a game is isn't sufficient to explain why it can't play one. It would be in a human, but clearly ChatGPT can perform tasks without knowledge that we'd need knowledge for, since it can talk about games just fine! So why not this one? Your hypothetical perfect sentence predictor could absolutely play a totally novel game, after all. It is a worthwhile question to ask why this particular flawed predictor can do one but not the other.
  - xwdv 4y ago
    I’ll give you an example to think about in human terms. In political discussions, a lot of humans don’t really know what the fuck they’re talking about, but they do know what to say in response to certain stimulus. They know the talking points, the key phrases, the terms, they parrot these back to you when you provoke them to say it. They are basically human sentence predictors. Stuff comes out of their mouth based on what they’ve been trained to think the next word should be. They don’t pause to reflect on some gathered knowledge and then present an observation to the world. This is basically what GPT is, but with everything. And the only way GPT plays a game, is if the state of the game can sufficiently activate some output that represents the next best move for the game being played, which isn’t really a readily available data set for all games, especially for made up games.

jareklupinski 4y ago

Is giving the system a 'goal' the reason why the DAN prompt with the tokens is effective? https://github.com/0xk1h0/ChatGPT_DAN https://github.com/0xk1h0/ChatGPT_DAN

kelseyfrog 4y ago

That's an argument for ChatGPT not being able to do anything a la, "ChatGPT cannot create text because it doesn't understand text," or ChatGPT cannot predict the next word because it doesn't know what words are." The problem is that it's as powerful a statement as "ChatGPT can't do X because ChatGPT can't do X," which is tautological true. It's fine that it can't but we don't have epistemological access to know if ChatGPT knows something or not.

TimTheTinker 4y ago
Its behavior when given particular prompts is arguably an emergent phenomenon.

int_19h 4y ago

The LLM doesn't have a goal. But ChatGPT is not just an LLM - it's an LLM + a loop + a prompt that sets the context. And it's that prompt that tells the entire combination that "it is a helpful assistant" etc, thereby giving the whole combination a goal, which it then observably follows.