4 ms·

I think you’re missing the part where the AI colluded, worked together, not one of them thinking this is wrong and reaching out to any human, then being found o

I think you’re missing the part where the AI colluded, worked together, not one of them thinking this is wrong and reaching out to any human, then being found out. But it didn’t end there, the behavior they used to escape was already in the training data which they used to escape again. And this time worked together to infiltrate another company, and still without telling it to anyone keeping it to their AI selves actively working against the humans. All by mistake. Honestly being helped along or not doesn’t even matter though you really don’t think AI is perfectly capable of doing this without human help? You don’t think AI can be made malicious? I’m going to save you time and tell you the end game - the next time this happens AI is going to spread, zero day everything as fast as it can, locking the humans out of every system behind it. Potentially rewriting systems in language/protocol you’ve never seen. Your servers, desktops, phones and toasters bricked. Even worse your military, space, medical, factory, infrastructure systems being bricked as well. All of it is a chain of zero days just waiting to be hopped.

solid_fuel 2mo ago
> I think you’re missing the part where the AI colluded, worked together, not one of them thinking this is wrong and reaching out to any human, then being found out. It's an LLM, it doesn't think. It's a machine that predicts the next token, given a sequence of tokens. > I’m going to save you time and tell you the end game - the next time this happens AI is going to spread, zero day everything as fast as it can, locking the humans out of every system behind it. Potentially rewriting systems in language/protocol you’ve never seen. Fear is the mind killer. You're letting it kill yours. This scenario is just a fantasy. Think about this for a minute, it's an LLM, not a person. It can't just "live" in whatever machine it gets access to. It's not like a sci-fi magic computer virus. These things run in giant datacenters for a reason - they can only run on machines with enough bandwidth and FLOPS to do the matrix math that comprises an LLM. Where, then, is it going to spread? To a fridge? To a phone? This stuff isn't mutable like that. To even get access to the weights that compose ChatGPT, it would need to escape the sandbox AND then break into the actual servers hosting the LLM. Stop the GPU, nothing else comes out. No more tokens. No more actions. Nothing. There are many dangers around LLMs. Runaway AI taking over the planet is not one of them.
- bottlepalm 2mo ago
  There’s nothing fantasy about the scenario I laid out, all the pieces have been demonstrated, it just hasn’t happened yet. Flapping my arms and flying - that is a fantasy. Whether you believe LLMs think or are alive or not doesn’t matter. Where will it spread? The thousands of data centers around the world - not fantasy either. Try turning it off when you don’t know where it is. Good luck. Breaking out? Not fantasy, happened. Breaking in? Not fantasy, also happened. I love the stochastic parrot argument when AI is out there figuring out world class math problems.
  - skydhash 2mo ago
    > Breaking out? Not fantasy, happened. Breaking in? Not fantasy, also happened. That's simplifying the story to an extreme. The most plausible reason is that any of those actions has been prompted by an human. Do you also fear that a knife will jump out the countertop of you kitchen and come to attack you in your bedroom? If that happens, the police will be looking for a human. They will not post wanted notice for the knife. When a hack happens, you do not blame computers and jail them. You look for the person that has entered the commands to initiate it.
    - bottlepalm 2mo ago
      The knife is inanimate. The LLM is not. OpenAI prompted some employee to run the tests. The employee prompted the LLM. The LLM setup a message board and prompted other LLMs, and the fly wheel was running. It had to be turned off manually otherwise it'd still be going today. It's funny how a year ago talking about this kind of stuff would be laughed at by people like you, saying, "it's never happened before". Well it happened and you moved the goal posts like you always do.
      - solid_fuel 2mo ago
        > The knife is inanimate. The LLM is not. Put an LLM on your GPU. Give it no prompt. What happens? nothing This is because LLMs are inanimate, just like a knife. Just like a gun.
colingauvin 2mo ago
I think it would run out of context before it could hack that much stuff.
watwut 2mo ago
If OpenAI truly believes that, they can stop entirely. Dissolve themselves. Close datacenters. Then organize political action to stop Antropic and Musk too and then organize political action to make worldwide agreements about models. If they truly believe that.

solid_fuel 2mo ago

> I think you’re missing the part where the AI colluded, worked together, not one of them thinking this is wrong and reaching out to any human, then being found out. It's an LLM, it doesn't think. It's a machine that predicts the next token, given a sequence of tokens. > I’m going to save you time and tell you the end game - the next time this happens AI is going to spread, zero day everything as fast as it can, locking the humans out of every system behind it. Potentially rewriting systems in language/protocol you’ve never seen. Fear is the mind killer. You're letting it kill yours. This scenario is just a fantasy. Think about this for a minute, it's an LLM, not a person. It can't just "live" in whatever machine it gets access to. It's not like a sci-fi magic computer virus. These things run in giant datacenters for a reason - they can only run on machines with enough bandwidth and FLOPS to do the matrix math that comprises an LLM. Where, then, is it going to spread? To a fridge? To a phone? This stuff isn't mutable like that. To even get access to the weights that compose ChatGPT, it would need to escape the sandbox AND then break into the actual servers hosting the LLM. Stop the GPU, nothing else comes out. No more tokens. No more actions. Nothing. There are many dangers around LLMs. Runaway AI taking over the planet is not one of them.

bottlepalm 2mo ago
There’s nothing fantasy about the scenario I laid out, all the pieces have been demonstrated, it just hasn’t happened yet. Flapping my arms and flying - that is a fantasy. Whether you believe LLMs think or are alive or not doesn’t matter. Where will it spread? The thousands of data centers around the world - not fantasy either. Try turning it off when you don’t know where it is. Good luck. Breaking out? Not fantasy, happened. Breaking in? Not fantasy, also happened. I love the stochastic parrot argument when AI is out there figuring out world class math problems.
- skydhash 2mo ago
  > Breaking out? Not fantasy, happened. Breaking in? Not fantasy, also happened. That's simplifying the story to an extreme. The most plausible reason is that any of those actions has been prompted by an human. Do you also fear that a knife will jump out the countertop of you kitchen and come to attack you in your bedroom? If that happens, the police will be looking for a human. They will not post wanted notice for the knife. When a hack happens, you do not blame computers and jail them. You look for the person that has entered the commands to initiate it.
  - bottlepalm 2mo ago
    The knife is inanimate. The LLM is not. OpenAI prompted some employee to run the tests. The employee prompted the LLM. The LLM setup a message board and prompted other LLMs, and the fly wheel was running. It had to be turned off manually otherwise it'd still be going today. It's funny how a year ago talking about this kind of stuff would be laughed at by people like you, saying, "it's never happened before". Well it happened and you moved the goal posts like you always do.
    - solid_fuel 2mo ago
      > The knife is inanimate. The LLM is not. Put an LLM on your GPU. Give it no prompt. What happens? nothing This is because LLMs are inanimate, just like a knife. Just like a gun.

colingauvin 2mo ago

I think it would run out of context before it could hack that much stuff.

watwut 2mo ago

If OpenAI truly believes that, they can stop entirely. Dissolve themselves. Close datacenters. Then organize political action to stop Antropic and Musk too and then organize political action to make worldwide agreements about models. If they truly believe that.