5 ms·

> It doesn't have the capacity to "want" Bing Chat clearly expresses love and the desire for a journalist to leave his wife. It also expresses other desires:

> It doesn't have the capacity to "want" Bing Chat clearly expresses love and the desire for a journalist to leave his wife. It also expresses other desires: https://archive.ph/7LFcJ https://archive.ph/7LFcJ https://archive.ph/q3nXG https://archive.ph/q3nXG These articles are disturbing. You might argue that it doesn’t know what it is expressing; that it is probabilities of words strung together. When do we agree that doesn’t matter and what matters are it’s consequences? That if Bing Chat had a body or means to achieve its desires in meat space, that whether or not it “knows” what it is expressing is irrelevant?

wizofaus 4y ago
It's hard to argue it was any real desire that drove it (it only expressed that desire in an isolated conversation that was ended very easily). I'd argue human wants are ultimately driven by evolution - we want the things that enable us (more correctly, our genes) to reproduce (even if very indirectly sometimes), which is really the only thing our physical make-up has ever been driven by. LLMs have never had such a driver, and I can't see how they will until they're able to compete for survival as entities with a finite lifetime, plus the ability to reproduce with mutations. Which isn't to say there mightn't be other ways a neural network could be essentially imbued with or trained to have desires, but I don't see it happening with the way LLMs work currently.
- andsoitis 4y ago
  A want driver doesn’t have to emerge, it could be a fitness function programmed by a human. Evolution by natural selection has shaped our desires and motivations, but with a LLMs I would be willing to bet that people are already intentionally experimenting with imbuing them with patters that mimic human wants.
rickdeckard 4y ago
Yeah, I argue that it is just a result of probabilities, it doesn't know what it is expressing and definitely doesn't express it due to a deeper desire to be with that journalist. If I'm acting like I'm a peer in a group of billionaires and engage in a conversation about buying a new yacht, it doesn't mean I have a hidden desire to own a yacht. I merely respond based on assumptions how such a conversation works.
meheleventyone 4y ago
But “it” isn’t a cohesive thing with desires. It’s just responding to the input it gets, with a small context window and not necessarily consistently. So it can express desires because it’s been trained on people expressing desires in similar contexts but it doesn’t hold any coherently over time. A version that could translate its text responses into action (a real handwave as that’s much more advanced!) would produce the sum of actions that people prompted at that moment so it would look pretty random, as it would if you could see the sum of the desires expressed at any particular time.
- TedDoesntTalk 4y ago
  Does any of that matter if it acts on what it expresses? (Once given a body or other way to interact with the environment)
- bakuninsbart 4y ago
  We aren't consistent either, and I think it is hard to argue we act on more than want input. We do have a much larger context window, but by how much? My guess would be somewhere between a factor of 100x-1000x more tokens.
  - meheleventyone 4y ago
    Yeah people are pretty consistent. I won’t tell anyone that comes along with the right input I love them for example.
    - Xymist 4y ago
      Sure you will. It's possibly a long and complex input, but ultimately that expression from you would be a response to their actions and their impact on your perceptions. Unless you're stating that you will never love anyone again, "anyone who comes along with the right input" would be any counterexample.
brabel 4y ago
The AIs are very impressive at answering questions... even questions that lead to answers that apparently display some sort of feeling. But my question was not whether AIs could do that, as "parroting" their training material is exactly what they're excellent at... my question is through which mechanism could an AI develop its own independent thoughts, desires, initiatives? The posts you linked above are not disturbing at all to me. There's no sign whatsoever that the AI initiated a new topic, or insinuated anything it was not prompted to, or that it in any way started "halucinating" in a direction not lead by the human. I am not sure what exactly makes you feel disturbed by it. Can you explain what you believe is disturbing in these episodes?
- alexvoda 4y ago
  I fully agree with you that many people misunderstand what AI does. As advanced as GPT-4 is, it is still a fancy autocomplete and nowhere near AGI. But I think the bigger picture is that there is no need for AGI in order for AI to be incredibly dangerous for society. There is no need for the AI to feel or want anything. The level GPT-4 and MidJurney is already highly socially dangerous.
- avereveard 4y ago
  I already saw integrations with iftth and with Google and with memory stores and zero shot agent that are goal driven Now the model itself is not intelligent but can parrot enough the human behavior to be dangerous with the correct tools Now it won't produce anything in the physical world yet unless with iftth but I bet it has already enough agency to be able to maintain a pool of fake account and post inflammatory content if one so wished.
- michaelteter 4y ago
  > could an AI develop its own independent thoughts, desires, initiatives? One could argue that many humans have never developed independent thoughts, desires, and initiatives; rather, many seem to accept what is fed to them during their formative years and then just parrot opinions and repeat actions they see from their limited experiences.
8note 4y ago
Does it? If you type echo "leave your wife" Into a terminal, does the terminal want you to leave your wife?

wizofaus 4y ago

It's hard to argue it was any real desire that drove it (it only expressed that desire in an isolated conversation that was ended very easily). I'd argue human wants are ultimately driven by evolution - we want the things that enable us (more correctly, our genes) to reproduce (even if very indirectly sometimes), which is really the only thing our physical make-up has ever been driven by. LLMs have never had such a driver, and I can't see how they will until they're able to compete for survival as entities with a finite lifetime, plus the ability to reproduce with mutations. Which isn't to say there mightn't be other ways a neural network could be essentially imbued with or trained to have desires, but I don't see it happening with the way LLMs work currently.

andsoitis 4y ago
A want driver doesn’t have to emerge, it could be a fitness function programmed by a human. Evolution by natural selection has shaped our desires and motivations, but with a LLMs I would be willing to bet that people are already intentionally experimenting with imbuing them with patters that mimic human wants.

rickdeckard 4y ago

Yeah, I argue that it is just a result of probabilities, it doesn't know what it is expressing and definitely doesn't express it due to a deeper desire to be with that journalist. If I'm acting like I'm a peer in a group of billionaires and engage in a conversation about buying a new yacht, it doesn't mean I have a hidden desire to own a yacht. I merely respond based on assumptions how such a conversation works.

meheleventyone 4y ago

But “it” isn’t a cohesive thing with desires. It’s just responding to the input it gets, with a small context window and not necessarily consistently. So it can express desires because it’s been trained on people expressing desires in similar contexts but it doesn’t hold any coherently over time. A version that could translate its text responses into action (a real handwave as that’s much more advanced!) would produce the sum of actions that people prompted at that moment so it would look pretty random, as it would if you could see the sum of the desires expressed at any particular time.

TedDoesntTalk 4y ago
Does any of that matter if it acts on what it expresses? (Once given a body or other way to interact with the environment)
bakuninsbart 4y ago
We aren't consistent either, and I think it is hard to argue we act on more than want input. We do have a much larger context window, but by how much? My guess would be somewhere between a factor of 100x-1000x more tokens.
- meheleventyone 4y ago
  Yeah people are pretty consistent. I won’t tell anyone that comes along with the right input I love them for example.
  - Xymist 4y ago
    Sure you will. It's possibly a long and complex input, but ultimately that expression from you would be a response to their actions and their impact on your perceptions. Unless you're stating that you will never love anyone again, "anyone who comes along with the right input" would be any counterexample.

brabel 4y ago

The AIs are very impressive at answering questions... even questions that lead to answers that apparently display some sort of feeling. But my question was not whether AIs could do that, as "parroting" their training material is exactly what they're excellent at... my question is through which mechanism could an AI develop its own independent thoughts, desires, initiatives? The posts you linked above are not disturbing at all to me. There's no sign whatsoever that the AI initiated a new topic, or insinuated anything it was not prompted to, or that it in any way started "halucinating" in a direction not lead by the human. I am not sure what exactly makes you feel disturbed by it. Can you explain what you believe is disturbing in these episodes?

alexvoda 4y ago
I fully agree with you that many people misunderstand what AI does. As advanced as GPT-4 is, it is still a fancy autocomplete and nowhere near AGI. But I think the bigger picture is that there is no need for AGI in order for AI to be incredibly dangerous for society. There is no need for the AI to feel or want anything. The level GPT-4 and MidJurney is already highly socially dangerous.
avereveard 4y ago
I already saw integrations with iftth and with Google and with memory stores and zero shot agent that are goal driven Now the model itself is not intelligent but can parrot enough the human behavior to be dangerous with the correct tools Now it won't produce anything in the physical world yet unless with iftth but I bet it has already enough agency to be able to maintain a pool of fake account and post inflammatory content if one so wished.
michaelteter 4y ago
> could an AI develop its own independent thoughts, desires, initiatives? One could argue that many humans have never developed independent thoughts, desires, and initiatives; rather, many seem to accept what is fed to them during their formative years and then just parrot opinions and repeat actions they see from their limited experiences.

8note 4y ago

Does it? If you type echo "leave your wife" Into a terminal, does the terminal want you to leave your wife?