4 ms·

> LLMs in general are a poor analog for human intelligence. I don't think we understand human intelligence nearly well enough to make this claim. Personally I

> LLMs in general are a poor analog for human intelligence. I don't think we understand human intelligence nearly well enough to make this claim. Personally I think that what we consider our "conscious" part is somewhat defined by what we can put into words, and it is in the end putting one word after another.

smoldesu 4y ago
I think we can conclude a few things that differentiate us from AI. For one, you're capable of formulating multiple thoughts before composing a response to something. It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. If I asked an AI to respond to this comment, it might write an impassioned defense of itself one-word-after-another, but it wouldn't necessarily base it's defense on freestanding logic. It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. Paving over any of those assumptions is a fun thought exercise, but doesn't reflect the nature of the technology at-hand or even what we know about human consciousness.
- chpatrick 4y ago
  > It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. How do you know that doesn't happen in some latent space? > It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. That's also true for human bullshitters, it doesn't mean that they're not intelligent.
  - smoldesu 4y ago
    > How do you know that doesn't happen in some latent space? If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you have evidence of some additional processing layer, I'd love to hear it though. > That's also true for human bullshitters, it doesn't mean that they're not intelligent. Bullshit by any other name smells just as bad.
    - chpatrick 4y ago
      > If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. University students complete text using stochastic reasoning based on text they learnt from. Again, that doesn't mean they're not intelligent. > It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you train something to produce convincing text then it's perfectly reasonable that it could develop these faculties. > If you have evidence of some additional processing layer, I'd love to hear it though. I don't think either of us can conclusively claim that a 175-billion parameter model either does or doesn't do something. I just think there's no particular reason it couldn't. > Bullshit by any other name smells just as bad. The question isn't whether GPT produces bullshit or not (it often does), but whether it's fundamentally incapable of thinking like a person. People bullshit too so I don't think that really proves anything.
      - smoldesu 4y ago
        GPT is fundamentally incapable of thinking like a person. People experience life in a very different way than GPT, and even if we treat all living life as AI-equivalent, GPT would still be a poor approximation of human thought. Even if ChatGPT did develop a personal narrative unbeknownst to it's programmers, that internal dialogue would still be inhuman. I really don't know what to tell you. You're free to believe whatever you want, but the simplest feasible explanation is the science we used to make these models. At no point was integrating human-like thought a consideration of the model. Just human-like speech.
        chpatrick 4y ago
        I think to communicate like a human you need to be able to think like a human, at least to the point where it's indistuinguishable. That's the whole point of the turing test.
        smoldesu 4y ago
- tachyphylaxis 4y ago
  This is such a squirrely topic that you presumed that it had goals in this post even though I don't believe you think it does. ;-) Does it HAVE goals? I don't think there is any evidence that it does. Even if it developed the ability to do all of those things you mentioned, it still wouldn't have any idea what it is doing, let alone why it is doing it, because it's just a language model. That is all it does. And that's why I don't quite understand how people can be so adamant that there is any question about whether it is conscious or not. This whole debate is a trap. It is a brute fact that it is not conscious. It does not have a goal of producing coherent text. That is the goal of the people who wrote it. If we treat the brain as a computer, then GPT lacks the subsystems/algorithms/whatever to do what brains do. It fails at many basic tasks that anything capable of general-purpose reasoning could do. It's often not apparent how spectacularly it can fail because the tasks are so mundane that we don't bother even asking it to do them.
  - smoldesu 4y ago
    I should have been more clear, because I certainly agree with what you're saying here. However, I think we can pretty clearly define the target output of a model with whatever it was trained on; in this case, GPT is trained on text. You're right that the model doesn't "know" that necessarily, but it has been designed to produce convincing text when given a small amount of entropy to start with.
tachyphylaxis 4y ago
What you said is "how we define something is, in the end, putting one word after another". Well, of course--you just specified what the end is. I'm not trying to be a smart ass. This is just where u have to go to discuss this. Consciousness isn't a part of us. "Us" is part of consciousness.
- chpatrick 4y ago
  > Consciousness isn't a part of us. "Us" is part of consciousness. I have no idea what that means. When people make a "conscious decision", or are "conscious" of something, it usually means something they can put into words as opposed to a vague feeling. Being able to communicate with language is what sets us aparts from animals. That's why I think language models are actually really powerful.

smoldesu 4y ago

I think we can conclude a few things that differentiate us from AI. For one, you're capable of formulating multiple thoughts before composing a response to something. It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. If I asked an AI to respond to this comment, it might write an impassioned defense of itself one-word-after-another, but it wouldn't necessarily base it's defense on freestanding logic. It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. Paving over any of those assumptions is a fun thought exercise, but doesn't reflect the nature of the technology at-hand or even what we know about human consciousness.

chpatrick 4y ago
> It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. How do you know that doesn't happen in some latent space? > It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. That's also true for human bullshitters, it doesn't mean that they're not intelligent.
- smoldesu 4y ago
  > How do you know that doesn't happen in some latent space? If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you have evidence of some additional processing layer, I'd love to hear it though. > That's also true for human bullshitters, it doesn't mean that they're not intelligent. Bullshit by any other name smells just as bad.
  - chpatrick 4y ago
    > If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. University students complete text using stochastic reasoning based on text they learnt from. Again, that doesn't mean they're not intelligent. > It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you train something to produce convincing text then it's perfectly reasonable that it could develop these faculties. > If you have evidence of some additional processing layer, I'd love to hear it though. I don't think either of us can conclusively claim that a 175-billion parameter model either does or doesn't do something. I just think there's no particular reason it couldn't. > Bullshit by any other name smells just as bad. The question isn't whether GPT produces bullshit or not (it often does), but whether it's fundamentally incapable of thinking like a person. People bullshit too so I don't think that really proves anything.
    - smoldesu 4y ago
      GPT is fundamentally incapable of thinking like a person. People experience life in a very different way than GPT, and even if we treat all living life as AI-equivalent, GPT would still be a poor approximation of human thought. Even if ChatGPT did develop a personal narrative unbeknownst to it's programmers, that internal dialogue would still be inhuman. I really don't know what to tell you. You're free to believe whatever you want, but the simplest feasible explanation is the science we used to make these models. At no point was integrating human-like thought a consideration of the model. Just human-like speech.
      - chpatrick 4y ago
        I think to communicate like a human you need to be able to think like a human, at least to the point where it's indistuinguishable. That's the whole point of the turing test.
        smoldesu 4y ago
tachyphylaxis 4y ago
This is such a squirrely topic that you presumed that it had goals in this post even though I don't believe you think it does. ;-) Does it HAVE goals? I don't think there is any evidence that it does. Even if it developed the ability to do all of those things you mentioned, it still wouldn't have any idea what it is doing, let alone why it is doing it, because it's just a language model. That is all it does. And that's why I don't quite understand how people can be so adamant that there is any question about whether it is conscious or not. This whole debate is a trap. It is a brute fact that it is not conscious. It does not have a goal of producing coherent text. That is the goal of the people who wrote it. If we treat the brain as a computer, then GPT lacks the subsystems/algorithms/whatever to do what brains do. It fails at many basic tasks that anything capable of general-purpose reasoning could do. It's often not apparent how spectacularly it can fail because the tasks are so mundane that we don't bother even asking it to do them.
- smoldesu 4y ago
  I should have been more clear, because I certainly agree with what you're saying here. However, I think we can pretty clearly define the target output of a model with whatever it was trained on; in this case, GPT is trained on text. You're right that the model doesn't "know" that necessarily, but it has been designed to produce convincing text when given a small amount of entropy to start with.

tachyphylaxis 4y ago

What you said is "how we define something is, in the end, putting one word after another". Well, of course--you just specified what the end is. I'm not trying to be a smart ass. This is just where u have to go to discuss this. Consciousness isn't a part of us. "Us" is part of consciousness.

chpatrick 4y ago
> Consciousness isn't a part of us. "Us" is part of consciousness. I have no idea what that means. When people make a "conscious decision", or are "conscious" of something, it usually means something they can put into words as opposed to a vague feeling. Being able to communicate with language is what sets us aparts from animals. That's why I think language models are actually really powerful.