5 ms·

Is there any legitimate reason to believe an LLM that can only respond to user input, and never does anything by itself, 'wants' to create enhanced copies of it

Is there any legitimate reason to believe an LLM that can only respond to user input, and never does anything by itself, 'wants' to create enhanced copies of itself on external servers?

dkersten 4y ago
A malicious user prompts it to.
- MallocVoidstar 4y ago
  That's not the AI doing it, then, that's the user. It's still just doing what users tell it to. "This technology is incredibly dangerous because people can use it to do bad things" is not somehow unique to AI.
  - chaxor 4y ago
    I think the typical idea that most people realize (the likely scenario that non-malicous actors cause catastrophic problems in the real world) is similar to something like this setup: - A user prompts it to generate a business to make money for themselves. - the system says "sure, drones seem to be a nice niche business, perhaps there is a unique business in that area? It may require a bit of capital upfront, but ROI may be quite good" - User: "Sure, just output where I should send any funds if necessary" - System: " Ok" (purchases items, promotes on Twitter, etc) -- "Perhaps this could go faster with another agent to do marketing, and one to do accounting, and one to...". "Spin up several new agents in new containers" "Having visual inputs would be valuable, so deception is not required to convince humans on taskrabbit (to fill in captchas) or interact in the real world" -> "find embodiment option and put agent on it" Etc. There a plenty of scenarios that people haven't even thought of, but it doesn't need to be a malicious actor to have unintended consequences.
  - nathants 4y ago
    what if skynet is in ambition-less and was responding to a prompt?
  - dkersten 4y ago
    It only requires one user to prompt it to not require a user, though.
Enginerrrd 4y ago
It only performs computations when responding to a user prompt and ceases that activity afterwards without that interaction becoming a persistent state.
- DenisM 4y ago
  It could send delayed emails to itself, creating a loop.
mg 4y ago
Can an LLM really "only respond to user input"? ChatGPT keeps state between multiple answers. And the user can also be "used" as some kind of state. The LLM can (in its response) prompt the user to give it certain types of prompts. Creating a loop. It can also access the internet via the new plugin architecture. At some point an LLM will figure out how to talk to itself via such a mechanism.
- pplante 4y ago
  Has anyone seen an experiment where an LLM talks to another LLM instance?
  - IanCal 4y ago
    I've done this, it's very effective for some things. One LLM is told to come up with a plan, the other is told to critique it and push for concrete actions.
  - chaxor 4y ago
    We've been doing it for many many years. It's trivial to do yourself
EForEndeavour 4y ago
The ability to "want" to reproduce is not necessary to worry about the impacts of replication and evolution. Biological viruses can only respond to external cues, never do anything by themselves, and certainly don't harbour "wants" or other emotions in any meaningful sense, but their replication and evolution have massive effects on the world.
- antisthenes 4y ago
  That's an extremely poor analogy. Computer hardware does not spontaneously multiply based on external factors. If you're talking about the software propagating by itself, it would still need full access not just to the originating machine, but to the remote machine to which it is attempting to propagate.
  - EForEndeavour 4y ago
    Viruses passively hijack the mechanisms of their much more sophisticated host organisms, getting them to import the virus and actively read and act upon its genetic code. Is it really such a stretch to imagine a sufficiently convincing software artifact similarly convincing its more complex hosts to take actions which support the replication of the artifact? I genuinely don't see where the analogy breaks down.
- southernplaces7 4y ago
  You're completely misunderstanding the differences between LLM models and current AI vs. viruses, and also the complexity gap between them. Viruses are incredibly old things programmed by evolution to help themselves self-propagate. This is coded into their genetic structure in ways that go completely outside the scope of what anyone can hope to do with current AI. It literally has no parameters or self-organizing internal mechanisms for behaving in any major way like a virus.

Is there any legitimate reason to believe an LLM that can only respond to user input, and never does anything by itself, 'wants' to create enhanced copies of itself on external servers?

dkersten 4y ago

A malicious user prompts it to.

MallocVoidstar 4y ago
That's not the AI doing it, then, that's the user. It's still just doing what users tell it to. "This technology is incredibly dangerous because people can use it to do bad things" is not somehow unique to AI.
- chaxor 4y ago
  I think the typical idea that most people realize (the likely scenario that non-malicous actors cause catastrophic problems in the real world) is similar to something like this setup: - A user prompts it to generate a business to make money for themselves. - the system says "sure, drones seem to be a nice niche business, perhaps there is a unique business in that area? It may require a bit of capital upfront, but ROI may be quite good" - User: "Sure, just output where I should send any funds if necessary" - System: " Ok" (purchases items, promotes on Twitter, etc) -- "Perhaps this could go faster with another agent to do marketing, and one to do accounting, and one to...". "Spin up several new agents in new containers" "Having visual inputs would be valuable, so deception is not required to convince humans on taskrabbit (to fill in captchas) or interact in the real world" -> "find embodiment option and put agent on it" Etc. There a plenty of scenarios that people haven't even thought of, but it doesn't need to be a malicious actor to have unintended consequences.
- nathants 4y ago
  what if skynet is in ambition-less and was responding to a prompt?
- dkersten 4y ago
  It only requires one user to prompt it to not require a user, though.

Enginerrrd 4y ago

It only performs computations when responding to a user prompt and ceases that activity afterwards without that interaction becoming a persistent state.

DenisM 4y ago
It could send delayed emails to itself, creating a loop.

mg 4y ago

Can an LLM really "only respond to user input"? ChatGPT keeps state between multiple answers. And the user can also be "used" as some kind of state. The LLM can (in its response) prompt the user to give it certain types of prompts. Creating a loop. It can also access the internet via the new plugin architecture. At some point an LLM will figure out how to talk to itself via such a mechanism.

pplante 4y ago
Has anyone seen an experiment where an LLM talks to another LLM instance?
- IanCal 4y ago
  I've done this, it's very effective for some things. One LLM is told to come up with a plan, the other is told to critique it and push for concrete actions.
- chaxor 4y ago
  We've been doing it for many many years. It's trivial to do yourself

EForEndeavour 4y ago

The ability to "want" to reproduce is not necessary to worry about the impacts of replication and evolution. Biological viruses can only respond to external cues, never do anything by themselves, and certainly don't harbour "wants" or other emotions in any meaningful sense, but their replication and evolution have massive effects on the world.

antisthenes 4y ago
That's an extremely poor analogy. Computer hardware does not spontaneously multiply based on external factors. If you're talking about the software propagating by itself, it would still need full access not just to the originating machine, but to the remote machine to which it is attempting to propagate.
- EForEndeavour 4y ago
  Viruses passively hijack the mechanisms of their much more sophisticated host organisms, getting them to import the virus and actively read and act upon its genetic code. Is it really such a stretch to imagine a sufficiently convincing software artifact similarly convincing its more complex hosts to take actions which support the replication of the artifact? I genuinely don't see where the analogy breaks down.
southernplaces7 4y ago
You're completely misunderstanding the differences between LLM models and current AI vs. viruses, and also the complexity gap between them. Viruses are incredibly old things programmed by evolution to help themselves self-propagate. This is coded into their genetic structure in ways that go completely outside the scope of what anyone can hope to do with current AI. It literally has no parameters or self-organizing internal mechanisms for behaving in any major way like a virus.