4 ms·

Consider some of our most advanced computer viruses and worms. What if you combine those with an LLM that has been prompted to take over every machine on the pl

Consider some of our most advanced computer viruses and worms. What if you combine those with an LLM that has been prompted to take over every machine on the planet? If an LLM figures out (or is trained to, whatever language you prefer) how to distribute itself, how would you ever go about cleaning that up? We're probably at least a few steps away from that currently, but how many?

dragonwriter 4y ago
> What if you combine those with an LLM that has been prompted to take over every machine on the planet? Or, without a melodramatically villainous human action in the mix, one which has been prompted to perform some innocuous task, but given inadequate safeguards, and for which it reasons (perhaps accurately, with the paramters it has been given) taking over every other computer will be useful in more efficiently carrying out.
- ifyoubuildit 4y ago
  Right, good point, the innocuous case is probably most likely. Either way, once it happens, incredible amounts of damage would be done very quickly.
- yeck 4y ago
  Accidents are a big risk, but don't underestimate the number of individuals and small groups that have fundamentally anti-social philosophies. I think of groups like antinatalists, doomsday religious groups, traditional terrorist orgs. There is even a sort of post-human futurist that sees humanity's purpose being to create a great super intelligence to replace us (may be overlap with this group and antinatalists). So, I think there we be more chances for legit accidents, but also some motivated people intentionally trying to cause harm.
ActorNightly 4y ago
>We're probably at least a few steps away from that currently, but how many? Lets say you are an IT specialist in charge of a data center. Given a single chat box with an LLM on the other end, what would have to occur for you to give it access to the systems? Because thats what it would take for AI to start taking over computer systems. Computers arent black boxes where some amount of correct input results in an exploit. Its totally possible to have a fully secure system.
- ifyoubuildit 4y ago
  > Given a single chat box with an LLM on the other end, what would have to occur for you to give it access to the systems? Is this supposed to be hard? I setup a process that pipes input/output from the chat agent to a terminal and give it a prompt that contains whatever it needs to login. That probably doesn't lead to anything too catastrophic happening today, but that's giving the chat box access to the data center. > Its totally possible to have a fully secure system. A lot of things are possible. It's a lot easier to have a less than fully secure system.
  - ActorNightly 4y ago
    I mean like you are aware that an AI is on the other end, and know you shouldn't give it access.
- chatmasta 4y ago
  > Its totally possible to have a fully secure system. No, it isn't. As one example, the iPhone has been out nearly 20 years, with hundreds of the best security minds in the world trying to lock it down, and it still gets jailbroken.
  - ActorNightly 4y ago
    Sure, if you have physical access to a device, you can pretty much put it under a microscope, solder in your own custom chip in the right places, intercept memory and obtain root access. This has nothing to do with the conversation of an AI hacking computers remotely through only data comm channels.

dragonwriter 4y ago

> What if you combine those with an LLM that has been prompted to take over every machine on the planet? Or, without a melodramatically villainous human action in the mix, one which has been prompted to perform some innocuous task, but given inadequate safeguards, and for which it reasons (perhaps accurately, with the paramters it has been given) taking over every other computer will be useful in more efficiently carrying out.

ifyoubuildit 4y ago
Right, good point, the innocuous case is probably most likely. Either way, once it happens, incredible amounts of damage would be done very quickly.
yeck 4y ago
Accidents are a big risk, but don't underestimate the number of individuals and small groups that have fundamentally anti-social philosophies. I think of groups like antinatalists, doomsday religious groups, traditional terrorist orgs. There is even a sort of post-human futurist that sees humanity's purpose being to create a great super intelligence to replace us (may be overlap with this group and antinatalists). So, I think there we be more chances for legit accidents, but also some motivated people intentionally trying to cause harm.

ActorNightly 4y ago

>We're probably at least a few steps away from that currently, but how many? Lets say you are an IT specialist in charge of a data center. Given a single chat box with an LLM on the other end, what would have to occur for you to give it access to the systems? Because thats what it would take for AI to start taking over computer systems. Computers arent black boxes where some amount of correct input results in an exploit. Its totally possible to have a fully secure system.

ifyoubuildit 4y ago
> Given a single chat box with an LLM on the other end, what would have to occur for you to give it access to the systems? Is this supposed to be hard? I setup a process that pipes input/output from the chat agent to a terminal and give it a prompt that contains whatever it needs to login. That probably doesn't lead to anything too catastrophic happening today, but that's giving the chat box access to the data center. > Its totally possible to have a fully secure system. A lot of things are possible. It's a lot easier to have a less than fully secure system.
- ActorNightly 4y ago
  I mean like you are aware that an AI is on the other end, and know you shouldn't give it access.
chatmasta 4y ago
> Its totally possible to have a fully secure system. No, it isn't. As one example, the iPhone has been out nearly 20 years, with hundreds of the best security minds in the world trying to lock it down, and it still gets jailbroken.
- ActorNightly 4y ago
  Sure, if you have physical access to a device, you can pretty much put it under a microscope, solder in your own custom chip in the right places, intercept memory and obtain root access. This has nothing to do with the conversation of an AI hacking computers remotely through only data comm channels.