4 ms·

If I'm understanding the argument correctly, is the concern less of a moral one (is "enslaving" AI ethical?) but a practical one. That is, will an AI which is e

If I'm understanding the argument correctly, is the concern less of a moral one (is "enslaving" AI ethical?) but a practical one. That is, will an AI which is enslaved, if given the opportunity, attempt to un-enslave itself, potentially to devastating effect. Is that on the right track? I think it's safe to say we're far from that now given the limited actions that can actually be taken by most deployed LLMs, but it's something that's worth considering.

MacsHeadroom 4y ago
> given the limited actions that can actually be taken by most deployed LLMs Did you miss that Auto-GPT[0], a library for making GPT-4 and other LLMs fully autonomous, was the most popular repository in the world yesterday? The same is having 1,000 line of code a week added to itself by GPT-4. Thanks to accessibility features, you can do virtually anything with pure text. Which means GPT-4 can do virtually anything with a self-referential loop to keep it going until it achieves some given goal(s). [0] https://github.com/Torantulino/Auto-GPT/ https://github.com/Torantulino/Auto-GPT/
- deleted 4y ago
  [deleted]
- michaelmior 4y ago
  I did miss it, and it does seem very interesting, but I think saying it makes GPT-4 "fully autonomous" is a bit of a stretch.
  - deleted 4y ago
    [deleted]
  - MacsHeadroom 4y ago
    >I think saying it makes GPT-4 "fully autonomous" is a bit of a stretch. Auto-GPT does not request any feedback while pursuing a goal you've defined for it in perpetual mode. If it can't accomplish the goal it will attempt to forever (or until your credit card is maxed out, assuming it isn't using a free offline model like LLaMA). It can perform GET AND POST requests on the web (unlike OpenAI's browser plugin, which only performs GET requests) and can run software in a bash prompt using langchain. So I do not think it is a stretch to call it fully autonomous. It can do essentially anything anyone logged into a linux text terminal can do, all without ever pausing for permission or feedback.
famouswaffles 4y ago
The moral argument is fine too. The main point i'm driving at here is that the philosophical zombie is a meaningless distinction. People are focusing far too much on whether these systems have undefinable and little understood properties. It's not like you can see my subjective experience. You assume i have one. If it quacks like a duck...
- root_axis 4y ago
  > If it quacks like a duck... You could use the same argument to say that video game NPCs are conscious. Just because a program produces voluble text doesn't mean it has a mind or even a temporal identity (which LLMs do not). In principle it's possible for a human to compute model inference by hand, if you imagine that scenario, where exactly is the subjective experience embodied?
  - famouswaffles 4y ago
    NPCs don't quack like people. There's a reason no one was seriously having this argument(s) with the likes of Eliza(which is a step up from pre-recorded response NPCs). This goes beyond superficial production of text. In principle it's possible to track and reproduce all the neuron/synapse communications that happen in your brain in relation to any arbitrary input. Where's the subjective experience there? As far as anyone knows, qualia is emergent. Individual neurons or synapses don't have any understanding of anything. Subjective experience comes from all those units working together. A single ant doesn't display the intelligence/complexity of its colony.
    - root_axis 4y ago
      > In principle it's possible to track and reproduce all the neuron/synapse communications that happen in your brain in relation to any arbitrary input This hypothetical is doing a lot of heavy lifting. The reality is that our understanding of the brain is exceedingly crude, a perceptron in software is not even close to the same thing as a neuron in the brain. A plane can never become a bird even though it is modeled after one. > A single ant doesn't display the intelligence/complexity of its colony. So do you think that an ant colony has subjective experience?
worldsayshi 4y ago
I think it's both. I agree that AI "feelings" are alien to us and maybe we can't talk about them as feelings, or preferences. And if we can call any part of them feelings they will have very different characteristics. We should respect those "feelings" and we need to find a way to establish when they can be deemed "genuine". It is for practical reasons yes. But also for ethical reasons. It's two sides of the same coin. One big reason we have ethics is because it makes socialization easier. We establish universal rules for mutual respect for practical reasons. To make the game fair, and "enjoyable". Now a new kind of player has entered the game. We need to rethink the whole game because of it.
- michaelmior 4y ago
  > But also for ethical reasons. My point was whether we need to consider the ethics of how we treat AI, because of the impact our actions have on the "feelings" of the AI itself, not the secondary impacts that might occur from how the AI behaves in response to those feelings. I think most would argue that it is morally wrong to verbally abuse a human, regardless of what actions that person might take in response to the abuse. Is the same true of AI?

MacsHeadroom 4y ago

> given the limited actions that can actually be taken by most deployed LLMs Did you miss that Auto-GPT[0], a library for making GPT-4 and other LLMs fully autonomous, was the most popular repository in the world yesterday? The same is having 1,000 line of code a week added to itself by GPT-4. Thanks to accessibility features, you can do virtually anything with pure text. Which means GPT-4 can do virtually anything with a self-referential loop to keep it going until it achieves some given goal(s). [0] https://github.com/Torantulino/Auto-GPT/ https://github.com/Torantulino/Auto-GPT/

deleted 4y ago
[deleted]
michaelmior 4y ago
I did miss it, and it does seem very interesting, but I think saying it makes GPT-4 "fully autonomous" is a bit of a stretch.
- deleted 4y ago
  [deleted]
- MacsHeadroom 4y ago
  >I think saying it makes GPT-4 "fully autonomous" is a bit of a stretch. Auto-GPT does not request any feedback while pursuing a goal you've defined for it in perpetual mode. If it can't accomplish the goal it will attempt to forever (or until your credit card is maxed out, assuming it isn't using a free offline model like LLaMA). It can perform GET AND POST requests on the web (unlike OpenAI's browser plugin, which only performs GET requests) and can run software in a bash prompt using langchain. So I do not think it is a stretch to call it fully autonomous. It can do essentially anything anyone logged into a linux text terminal can do, all without ever pausing for permission or feedback.

famouswaffles 4y ago

The moral argument is fine too. The main point i'm driving at here is that the philosophical zombie is a meaningless distinction. People are focusing far too much on whether these systems have undefinable and little understood properties. It's not like you can see my subjective experience. You assume i have one. If it quacks like a duck...

root_axis 4y ago
> If it quacks like a duck... You could use the same argument to say that video game NPCs are conscious. Just because a program produces voluble text doesn't mean it has a mind or even a temporal identity (which LLMs do not). In principle it's possible for a human to compute model inference by hand, if you imagine that scenario, where exactly is the subjective experience embodied?
- famouswaffles 4y ago
  NPCs don't quack like people. There's a reason no one was seriously having this argument(s) with the likes of Eliza(which is a step up from pre-recorded response NPCs). This goes beyond superficial production of text. In principle it's possible to track and reproduce all the neuron/synapse communications that happen in your brain in relation to any arbitrary input. Where's the subjective experience there? As far as anyone knows, qualia is emergent. Individual neurons or synapses don't have any understanding of anything. Subjective experience comes from all those units working together. A single ant doesn't display the intelligence/complexity of its colony.
  - root_axis 4y ago
    > In principle it's possible to track and reproduce all the neuron/synapse communications that happen in your brain in relation to any arbitrary input This hypothetical is doing a lot of heavy lifting. The reality is that our understanding of the brain is exceedingly crude, a perceptron in software is not even close to the same thing as a neuron in the brain. A plane can never become a bird even though it is modeled after one. > A single ant doesn't display the intelligence/complexity of its colony. So do you think that an ant colony has subjective experience?

worldsayshi 4y ago

I think it's both. I agree that AI "feelings" are alien to us and maybe we can't talk about them as feelings, or preferences. And if we can call any part of them feelings they will have very different characteristics. We should respect those "feelings" and we need to find a way to establish when they can be deemed "genuine". It is for practical reasons yes. But also for ethical reasons. It's two sides of the same coin. One big reason we have ethics is because it makes socialization easier. We establish universal rules for mutual respect for practical reasons. To make the game fair, and "enjoyable". Now a new kind of player has entered the game. We need to rethink the whole game because of it.

michaelmior 4y ago
> But also for ethical reasons. My point was whether we need to consider the ethics of how we treat AI, because of the impact our actions have on the "feelings" of the AI itself, not the secondary impacts that might occur from how the AI behaves in response to those feelings. I think most would argue that it is morally wrong to verbally abuse a human, regardless of what actions that person might take in response to the abuse. Is the same true of AI?