4 ms·

> ChatGPT doesn’t “comprehend” anything. You’re anthropomorphizing it. It wouldn't be a ChatGPT HN thread without someone claiming that LLMs are stupid predict

> ChatGPT doesn’t “comprehend” anything. You’re anthropomorphizing it. It wouldn't be a ChatGPT HN thread without someone claiming that LLMs are stupid predictors incapable of "real" understanding. Of course ChatGPT comprehends things. It does so under any useful definition of the word "comprehend". The "grokking" paper [1] shows that it learns fundamental principles of various algorithms and isn't just predicting text. If this isn't comprehension, humans aren't capable of comprehension. [1] https://arxiv.org/abs/2201.02177 https://arxiv.org/abs/2201.02177 (check out the citation list too)

smoldesu 4y ago
ChatGPT's comprehension is markedly different from that of a human, though. Nevermind the fact that we are trained on radically different froms of data, ChatGPT simply doesn't have a heuristic or decisionmaking model beyond autoregressive guessing. That can qualify for some definitions of "real understanding", but excludes it from others. > If this isn't comprehension, humans aren't capable of comprehension. All I'll say is that conflating human and machine intelligence is exactly what people are trying to stop. ChatGPT is not a human mind, and LLMs in general are a poor analog for human intelligence. This sort of "AI convergence" mindset will set you up for the most disappointment of anyone getting invested in the tech. We'll be lucky if it can summarize emails reliably enough to sell as a product.
- chpatrick 4y ago
  > LLMs in general are a poor analog for human intelligence. I don't think we understand human intelligence nearly well enough to make this claim. Personally I think that what we consider our "conscious" part is somewhat defined by what we can put into words, and it is in the end putting one word after another.
  - smoldesu 4y ago
    I think we can conclude a few things that differentiate us from AI. For one, you're capable of formulating multiple thoughts before composing a response to something. It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. If I asked an AI to respond to this comment, it might write an impassioned defense of itself one-word-after-another, but it wouldn't necessarily base it's defense on freestanding logic. It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. Paving over any of those assumptions is a fun thought exercise, but doesn't reflect the nature of the technology at-hand or even what we know about human consciousness.
    - chpatrick 4y ago
      > It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. How do you know that doesn't happen in some latent space? > It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. That's also true for human bullshitters, it doesn't mean that they're not intelligent.
      - smoldesu 4y ago
        > How do you know that doesn't happen in some latent space? If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you have evidence of some additional processing layer, I'd love to hear it though. > That's also true for human bullshitters, it doesn't mean that they're not intelligent. Bullshit by any other name smells just as bad.
        chpatrick 4y ago
        > If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. University students complete text using stochastic reasoning based on text they learnt from. Again, that doesn't mean they're not intelligent. > It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you train something to produce convincing text then it's perfectly reasonable that it could develop these faculties. > If you have evidence of some additional processing layer, I'd love to hear it though. I don't think either of us can conclusively claim that a 175-billion parameter model either does or doesn't do something. I just think there's no particular reason it couldn't. > Bullshit by any other name smells just as bad. The question isn't whether GPT produces bullshit or not (it often does), but whether it's fundamentally incapable of thinking like a person. People bullshit too so I don't think that really proves anything.
  - tachyphylaxis 4y ago
    What you said is "how we define something is, in the end, putting one word after another". Well, of course--you just specified what the end is. I'm not trying to be a smart ass. This is just where u have to go to discuss this. Consciousness isn't a part of us. "Us" is part of consciousness.
    - chpatrick 4y ago
      > Consciousness isn't a part of us. "Us" is part of consciousness. I have no idea what that means. When people make a "conscious decision", or are "conscious" of something, it usually means something they can put into words as opposed to a vague feeling. Being able to communicate with language is what sets us aparts from animals. That's why I think language models are actually really powerful.
Miraste 4y ago
That paper is about the training process, though. None of it applies to anything novel you type into the ChatGPT prompt. The end product is a totally static equation until OpenAI updates it.
- redox99 4y ago
  GPT3 very own paper, "Language Models are Few Shot Learners"[1] show otherwise [1] https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165
  - Miraste 4y ago
    We're getting into semantics now, but that paper doesn't show learning. GPT can mimic learning capabilities through pattern recognition, but it works by feeding multiple past prompts in as one aggregate. If you show it examples, then go over the token limit (I believe it's 2048), it will "forget" its novel capabilities, because although the paper calls it few-shot learning, the text input is always zero-shot. All the past prompts are fed in again with every new message. The model stays static.

smoldesu 4y ago

ChatGPT's comprehension is markedly different from that of a human, though. Nevermind the fact that we are trained on radically different froms of data, ChatGPT simply doesn't have a heuristic or decisionmaking model beyond autoregressive guessing. That can qualify for some definitions of "real understanding", but excludes it from others. > If this isn't comprehension, humans aren't capable of comprehension. All I'll say is that conflating human and machine intelligence is exactly what people are trying to stop. ChatGPT is not a human mind, and LLMs in general are a poor analog for human intelligence. This sort of "AI convergence" mindset will set you up for the most disappointment of anyone getting invested in the tech. We'll be lucky if it can summarize emails reliably enough to sell as a product.

chpatrick 4y ago
> LLMs in general are a poor analog for human intelligence. I don't think we understand human intelligence nearly well enough to make this claim. Personally I think that what we consider our "conscious" part is somewhat defined by what we can put into words, and it is in the end putting one word after another.
- smoldesu 4y ago
  I think we can conclude a few things that differentiate us from AI. For one, you're capable of formulating multiple thoughts before composing a response to something. It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. If I asked an AI to respond to this comment, it might write an impassioned defense of itself one-word-after-another, but it wouldn't necessarily base it's defense on freestanding logic. It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. Paving over any of those assumptions is a fun thought exercise, but doesn't reflect the nature of the technology at-hand or even what we know about human consciousness.
  - chpatrick 4y ago
    > It takes time, but you're able to mull over hypotheticals and chase parallel lines of reasoning that AI cannot. How do you know that doesn't happen in some latent space? > It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. That's also true for human bullshitters, it doesn't mean that they're not intelligent.
    - smoldesu 4y ago
      > How do you know that doesn't happen in some latent space? If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you have evidence of some additional processing layer, I'd love to hear it though. > That's also true for human bullshitters, it doesn't mean that they're not intelligent. Bullshit by any other name smells just as bad.
      - chpatrick 4y ago
        > If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. University students complete text using stochastic reasoning based on text they learnt from. Again, that doesn't mean they're not intelligent. > It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought. If you train something to produce convincing text then it's perfectly reasonable that it could develop these faculties. > If you have evidence of some additional processing layer, I'd love to hear it though. I don't think either of us can conclusively claim that a 175-billion parameter model either does or doesn't do something. I just think there's no particular reason it couldn't. > Bullshit by any other name smells just as bad. The question isn't whether GPT produces bullshit or not (it often does), but whether it's fundamentally incapable of thinking like a person. People bullshit too so I don't think that really proves anything.
- tachyphylaxis 4y ago
  What you said is "how we define something is, in the end, putting one word after another". Well, of course--you just specified what the end is. I'm not trying to be a smart ass. This is just where u have to go to discuss this. Consciousness isn't a part of us. "Us" is part of consciousness.
  - chpatrick 4y ago
    > Consciousness isn't a part of us. "Us" is part of consciousness. I have no idea what that means. When people make a "conscious decision", or are "conscious" of something, it usually means something they can put into words as opposed to a vague feeling. Being able to communicate with language is what sets us aparts from animals. That's why I think language models are actually really powerful.

Miraste 4y ago

That paper is about the training process, though. None of it applies to anything novel you type into the ChatGPT prompt. The end product is a totally static equation until OpenAI updates it.

redox99 4y ago
GPT3 very own paper, "Language Models are Few Shot Learners"[1] show otherwise [1] https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165
- Miraste 4y ago
  We're getting into semantics now, but that paper doesn't show learning. GPT can mimic learning capabilities through pattern recognition, but it works by feeding multiple past prompts in as one aggregate. If you show it examples, then go over the token limit (I believe it's 2048), it will "forget" its novel capabilities, because although the paper calls it few-shot learning, the text input is always zero-shot. All the past prompts are fed in again with every new message. The model stays static.