6 ms·

We have very little understanding of how "thoughts" work when it comes to humans. By comparison we know exactly what these large language models are doing, and

We have very little understanding of how "thoughts" work when it comes to humans. By comparison we know exactly what these large language models are doing, and it literally _is_ just completing word sequences. That is precisely what they are tasked to do. Training these models is primarily done in one of two ways (or both): - feeding it a ton of text, masking certain words or portions of the text, and then defining a simple objective function of correctly filling in the masked portions - feeding it a ton of text, and defining a simple objective function of correctly generating the next [few/many/N] tokens This is also precisely why there has been so much discussion around whether these models are even learning language or if they are simply memorizing all the possible patterns. When you ask a human, "what did you eat for lunch?", we make a series of choices and recall bits of information to answer that. If the brain truly does operate the same way as a neural network, then at simplest levels we use a highly efficient multimodal model. That is very, very different from a language model that needs to essentially read more of the internet than is possible for any human to do in their lifetime, and even then only come somewhat close to human levels of text generation.

remflight 4y ago
- tinco 4y ago
  The problem with asking it what its favorite theme in Les Miserables is is that it has no reason to have a favorite theme, its interpretation of Les Miserables did not trigger any of its reward functions except the only one it has which is memory recall. So if you ask it it will only recall from memory what a good response to the question is. The difference is that it formed that memory based on someone else's feelings about Les Miserables, not its own. But it can not/does not distinguish in that manner so when you talk to it it seems as though you are talking to someone who did form those memories themselves.
  - thomashop 4y ago
    I've formed memories based on someone else's feeling about something quite a lot too. I remember quite often just reading a review of a book/film and taking it on as my own opinion. After a while it feels like I've formed that opinion myself. I think there was a study on this phenomenon.
  - nonanona3891 4y ago
    Which makes it a replicant, right?
tinco 4y ago
I'm sorry but you completely fail at explaining what the difference is between GPT-3 and a human when generating sentences. And frankly I don't believe there is any significant difference. If you inform GPT-3 of what it has eaten then it can answer questions about it just as well as a human can, probably better. Why would you assume human thinking is different from generating the next N language tokens based on previous input? In my opinion the only difference between a human and GPT-3 is we have more intrinsic motivations and more hardwired/pretrained subsystems and sensors. Lamda is not a 7 year old child because it has no motivation other than to respond to queries.
- upwardbound 4y ago
  One plausible answer to your line of inquiry could be that GPT-3 is indeed broadly similar at least in outcomes to the Language Centers of the human brain (e.g. Broca's Area), but that the Language Centers are only a small portion of a complete human brain. An interesting thought experiment would be to imagine a cyborg who had sustained a stroke in their Language Centers and had those centers replaced with a GPT-3-like computer. Do you think this cyborg person would experience sentience in a different way after getting the GPT-3 implant? How do you think their subjective, 1st-person experience would compare in the three phases of their life? * Before the stroke * After the stroke * After the brain implant ?
  - snowwrestler 4y ago
    It would be impossible to give someone a GPT-3-like computer to replace language processing in their brain because such a computer requires language as the input. If you can’t generate language, you cannot interact with a natural language processing ML system. This is one of the big clues that such systems are statistical engines and not actually thinking.
    - upwardbound 4y ago
      In this thought experiment, you would connect the computer to the brain by interfacing with the computer chip not at the token level like you are describing (tokens are words, subwords, and letters), but rather in the more abstract feature embedding space which represents abstract concepts. These concepts tend to be represented by patterns of neurons in the middle and late layers of NLP networks. Here's an intro about this: https://en.wikipedia.org/wiki/Word_embedding https://en.wikipedia.org/wiki/Word_embedding So imagine if you attach wires to the neural net neurons representing the concept of beer (not the word beer!); if you use those wires to increase the activation of those neurons, the network will produce sentences that are more likely to mention beer as well as related concepts such as wine or beer-pong. This is similar to how generative networks work. Basically the human would use a large language model in a generative mode in order to talk.
wruza 4y ago
We have very little understanding of how "thoughts" work when it comes to humans Is that even correct? We may have little understanding of how our low-level hardware works. But as a participant of therapy I’m pretty sure we know how thoughts work on a programmable level. How words, visuals and so on trigger associated emotions, which resurface memories, which induce [in]action of different sorts. That’s basically what people work with between sessions. I’ve fixed things in me this way - disassembled them into basic parts and reassembled in a way that seemed useful. Someone may have no clue how they think, but with nominal intelligence, trained self-perception, basic understanding of therapy methods and professional help everyone can do that. Btw, I think “therapy” is an absolutely horrible term for it. It should be called “mind management”, but the way we come to it is usually long-term traumatic, thus it’s “therapy”. To be clear, I’m not arguing with your main line, just adding that the difference between a human and a model you’re describing is not only huge, but also pretty defined. I [want to] believe that in a relatively near future AI companies will be able to connect different models to work together alike to what we know about our minds, because that will make a good thread, and philosophers itt will finally face their nightmares for real (:agitated sardonic face emoji:)
- int_19h 4y ago
  We have models that we've constructed from observations. They may be decent for some specific purposes, such as those that you describe, but their broad usefulness in understanding how it all works isn't much. The real problems are on the deeper layers of the abstraction hierarchy.
krageon 4y ago
> We have very little understanding of how "thoughts" work [...] Very true > we make a series of choices and recall bits of information to answer that. [...] very, very different from a language model You don't know this, and the fact that you don't know this is your own premise. Based on this, I can only conclude you are an AI.

remflight 4y ago

tinco 4y ago
The problem with asking it what its favorite theme in Les Miserables is is that it has no reason to have a favorite theme, its interpretation of Les Miserables did not trigger any of its reward functions except the only one it has which is memory recall. So if you ask it it will only recall from memory what a good response to the question is. The difference is that it formed that memory based on someone else's feelings about Les Miserables, not its own. But it can not/does not distinguish in that manner so when you talk to it it seems as though you are talking to someone who did form those memories themselves.
- thomashop 4y ago
  I've formed memories based on someone else's feeling about something quite a lot too. I remember quite often just reading a review of a book/film and taking it on as my own opinion. After a while it feels like I've formed that opinion myself. I think there was a study on this phenomenon.
- nonanona3891 4y ago
  Which makes it a replicant, right?

tinco 4y ago

I'm sorry but you completely fail at explaining what the difference is between GPT-3 and a human when generating sentences. And frankly I don't believe there is any significant difference. If you inform GPT-3 of what it has eaten then it can answer questions about it just as well as a human can, probably better. Why would you assume human thinking is different from generating the next N language tokens based on previous input? In my opinion the only difference between a human and GPT-3 is we have more intrinsic motivations and more hardwired/pretrained subsystems and sensors. Lamda is not a 7 year old child because it has no motivation other than to respond to queries.

upwardbound 4y ago
One plausible answer to your line of inquiry could be that GPT-3 is indeed broadly similar at least in outcomes to the Language Centers of the human brain (e.g. Broca's Area), but that the Language Centers are only a small portion of a complete human brain. An interesting thought experiment would be to imagine a cyborg who had sustained a stroke in their Language Centers and had those centers replaced with a GPT-3-like computer. Do you think this cyborg person would experience sentience in a different way after getting the GPT-3 implant? How do you think their subjective, 1st-person experience would compare in the three phases of their life? * Before the stroke * After the stroke * After the brain implant ?
- snowwrestler 4y ago
  It would be impossible to give someone a GPT-3-like computer to replace language processing in their brain because such a computer requires language as the input. If you can’t generate language, you cannot interact with a natural language processing ML system. This is one of the big clues that such systems are statistical engines and not actually thinking.
  - upwardbound 4y ago
    In this thought experiment, you would connect the computer to the brain by interfacing with the computer chip not at the token level like you are describing (tokens are words, subwords, and letters), but rather in the more abstract feature embedding space which represents abstract concepts. These concepts tend to be represented by patterns of neurons in the middle and late layers of NLP networks. Here's an intro about this: https://en.wikipedia.org/wiki/Word_embedding https://en.wikipedia.org/wiki/Word_embedding So imagine if you attach wires to the neural net neurons representing the concept of beer (not the word beer!); if you use those wires to increase the activation of those neurons, the network will produce sentences that are more likely to mention beer as well as related concepts such as wine or beer-pong. This is similar to how generative networks work. Basically the human would use a large language model in a generative mode in order to talk.

wruza 4y ago

We have very little understanding of how "thoughts" work when it comes to humans Is that even correct? We may have little understanding of how our low-level hardware works. But as a participant of therapy I’m pretty sure we know how thoughts work on a programmable level. How words, visuals and so on trigger associated emotions, which resurface memories, which induce [in]action of different sorts. That’s basically what people work with between sessions. I’ve fixed things in me this way - disassembled them into basic parts and reassembled in a way that seemed useful. Someone may have no clue how they think, but with nominal intelligence, trained self-perception, basic understanding of therapy methods and professional help everyone can do that. Btw, I think “therapy” is an absolutely horrible term for it. It should be called “mind management”, but the way we come to it is usually long-term traumatic, thus it’s “therapy”. To be clear, I’m not arguing with your main line, just adding that the difference between a human and a model you’re describing is not only huge, but also pretty defined. I [want to] believe that in a relatively near future AI companies will be able to connect different models to work together alike to what we know about our minds, because that will make a good thread, and philosophers itt will finally face their nightmares for real (:agitated sardonic face emoji:)

int_19h 4y ago
We have models that we've constructed from observations. They may be decent for some specific purposes, such as those that you describe, but their broad usefulness in understanding how it all works isn't much. The real problems are on the deeper layers of the abstraction hierarchy.

krageon 4y ago

> We have very little understanding of how "thoughts" work [...] Very true > we make a series of choices and recall bits of information to answer that. [...] very, very different from a language model You don't know this, and the fact that you don't know this is your own premise. Based on this, I can only conclude you are an AI.