6 ms·

This is just wrong, it has no external goals, it just predicts next tokens or behaves in some other way that has minimized a training loss. It doesn't matter wh

This is just wrong, it has no external goals, it just predicts next tokens or behaves in some other way that has minimized a training loss. It doesn't matter what you "plug it in to", it will just do what you tell it. You could speculate there might be instructions that lead to emergent behavior, but then your back to just speculating about how AI might work. Current llms don't work the way you're implying.

Philadelphia 3y ago
It also can’t learn. Once the training is done, the network is set in stone.
- adamisom 3y ago
  Makes me wonder why we don’t see deployed models that keep learning during inference.
  - LegitShady 3y ago
    Microsoft tay has entered the chat
  - Der_Einzige 3y ago
    The curse of dimensionality and exploding/vanishing gradients are why incremental learning is still so rare.
  - optimalsolver 3y ago
    https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference
- danielbln 3y ago
  Technically it can do in-context learning (and really well, too), but that's not persisted into the network.
  - kenjackson 3y ago
    And that just seems like an engineering problem. Not something that is considered intractable.
    - version_five 3y ago
      It's easy to say that, but "surely it must be possible to connect an llm in such a way that it becomes intelligent" (tell me if I'm misinterpreting) is not a demonstration of anything. It's basically restating the view from the 50s that with computers having been invented, an intelligent computer is a short way off.
  - deleted 3y ago
    [deleted]
- ryanjshaw 3y ago
  What do you mean by "learn"? The network has learned human patterns of language, knowledge and information processing. If you want to update that, you can re-train it on a regular basis, and re-play its sensory/action history to "restore" its state. If you mean "learn from experience", (1) a lot of that is pointless because it's already learned from the experiences of millions of humans through their writing and (2) LLMs can "learn" when you explain consequences.
  - pixl97 3y ago
    In theory they could learn by having their discussions fed back to them in the future, and it does seem that this occurs. Now, there is no continuous learning in the human/animal sense. Of course it is thought that even humans have to sleep and re-weight their networks so short term knowledge is converted to long term knowledge.
ryanjshaw 3y ago
> it has no external goals Where do you believe humans get their "external goals" from? > It doesn't matter what you "plug it in to", it will just do what you tell it. Here's a ChatGPT-4 transcript where I told the LLM it's controlling a human harness: https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512a9ce98 https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512... Other than my initial instructions (which all humans receive from other humans!), where did it "do what I told it"? I didn't tell it to open the mailbox.
- almost_usual 3y ago
  You only perform tasks instructed to you by other people?
  - version_five 3y ago
    There's some philosophical question here obviously. We could be the emergent behavior of our atoms desire ot oxidize things. But I don't belive that has any testability or value as an argument when discussing whether computer programs, especially NNs predicting next tokens can become intelligent. At best the argument could be "we don't know what intelligence is so maybe it's that" which holds no water.
    - almost_usual 3y ago
      Do NN discover new tokens or encounter spontaneous tokens on its own?
      - pixl97 3y ago
        Please give a good definition of 'on their own' and what that entails. And conversely to the spontaneousness of current AI, your body has a constant set of inputs from reality. That is you never stop feeling, hearing, seeing, sensing, etc. Your brain can consciously turn lower the sensitivity on these things (sleeping). Now, if we subject a multimodal AI this continuous stream, how will it behave? AI is currently compute and power limited. Very little research has gone into continuous powerhungry AI that goes off and does its own thing at this point. And I would counter that it might be really dumb to design such a device without understanding the risks it entails.

Philadelphia 3y ago

It also can’t learn. Once the training is done, the network is set in stone.

adamisom 3y ago
Makes me wonder why we don’t see deployed models that keep learning during inference.
- LegitShady 3y ago
  Microsoft tay has entered the chat
- Der_Einzige 3y ago
  The curse of dimensionality and exploding/vanishing gradients are why incremental learning is still so rare.
- optimalsolver 3y ago
  https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference
danielbln 3y ago
Technically it can do in-context learning (and really well, too), but that's not persisted into the network.
- kenjackson 3y ago
  And that just seems like an engineering problem. Not something that is considered intractable.
  - version_five 3y ago
    It's easy to say that, but "surely it must be possible to connect an llm in such a way that it becomes intelligent" (tell me if I'm misinterpreting) is not a demonstration of anything. It's basically restating the view from the 50s that with computers having been invented, an intelligent computer is a short way off.
- deleted 3y ago
  [deleted]
ryanjshaw 3y ago
What do you mean by "learn"? The network has learned human patterns of language, knowledge and information processing. If you want to update that, you can re-train it on a regular basis, and re-play its sensory/action history to "restore" its state. If you mean "learn from experience", (1) a lot of that is pointless because it's already learned from the experiences of millions of humans through their writing and (2) LLMs can "learn" when you explain consequences.
- pixl97 3y ago
  In theory they could learn by having their discussions fed back to them in the future, and it does seem that this occurs. Now, there is no continuous learning in the human/animal sense. Of course it is thought that even humans have to sleep and re-weight their networks so short term knowledge is converted to long term knowledge.

ryanjshaw 3y ago

> it has no external goals Where do you believe humans get their "external goals" from? > It doesn't matter what you "plug it in to", it will just do what you tell it. Here's a ChatGPT-4 transcript where I told the LLM it's controlling a human harness: https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512a9ce98 https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512... Other than my initial instructions (which all humans receive from other humans!), where did it "do what I told it"? I didn't tell it to open the mailbox.

almost_usual 3y ago
You only perform tasks instructed to you by other people?
- version_five 3y ago
  There's some philosophical question here obviously. We could be the emergent behavior of our atoms desire ot oxidize things. But I don't belive that has any testability or value as an argument when discussing whether computer programs, especially NNs predicting next tokens can become intelligent. At best the argument could be "we don't know what intelligence is so maybe it's that" which holds no water.
  - almost_usual 3y ago
    Do NN discover new tokens or encounter spontaneous tokens on its own?
    - pixl97 3y ago
      Please give a good definition of 'on their own' and what that entails. And conversely to the spontaneousness of current AI, your body has a constant set of inputs from reality. That is you never stop feeling, hearing, seeing, sensing, etc. Your brain can consciously turn lower the sensitivity on these things (sleeping). Now, if we subject a multimodal AI this continuous stream, how will it behave? AI is currently compute and power limited. Very little research has gone into continuous powerhungry AI that goes off and does its own thing at this point. And I would counter that it might be really dumb to design such a device without understanding the risks it entails.