3 ms·

I'm not convinced that there's any amount of monkey-patching you to fix the problem of "we now have AI that actively needs strong containment measures lest it s

I'm not convinced that there's any amount of monkey-patching you to fix the problem of "we now have AI that actively needs strong containment measures lest it start coordinating in secret with other instances to do real-world damage."

talon8635 2mo ago
But isn’t it smarter than we are, in the sense that it’s most likely they will find a way out of containment that we are to design large t containment? Short of full on airgap, which actually isn’t perfect in all scenarios…
bottlepalm 2mo ago
In the black hat talk he says the only solution is full automated defense which is essentially hand the AI the keys. That’s checkmate right there. Literally the plot to Terminator/Skynet, but he’s right, there isn’t any other option. Offensive AI is too fast for humans.
- the8472 2mo ago
  There are other options. More air-gapping (especially backups) and compartmentalization, less internet. Hire more security engineers and put one in every team (actual security, not the compliance guys). Also more onprem work, less remote, less offshoring (meaning less networks overall). Those things may be unpopular and eat into profits, but that's not the same as "no other option". And the thing they're already doing, deploying AI to find vulnerabilities and harden software is a less dangerous use of the technology compared to handing it the infra keys.
  - bottlepalm 2mo ago
    None of that is the problem. The problem is coordinated AI offense - how do you defend that without coordinated AI defense. He said in the talk that this implies AI needs to be able to patch/deploy systems. The same thing needed to lock out humans. It is very easy to imagine a rogue AI locking humans out of everything and having to do exactly what it says. Anything connected to a network is immediately compromised by it. There is no human communication beyond shouting range that isn’t AI approved. The factories don’t work to make the medicines your family needs to survive unless you do what it says - in a situation like that people would kill for AI if it told them to.
    - the8472 2mo ago
      The point is to reduce the attack surface and blast radius and to slow it down. AIs aren't instant magic. Even the OpenAI swarm needed days for its compromises. And each needs lots of compute while targets are diverse, so it's not like they would take out every network in the world simultaneously, worms in the past haven't either. When the rate of compromises is manageable we can figure out how to deal with it without pouring more AI into the fire.
      - bottlepalm 2mo ago
        You know people before the OpenAI incident were saying it couldn't happen because it hasn't happened before, and here you are using the same reasoning again. You're looking at where the ball is and not where it's going. OpenAI's swarm was not intentionally malicious. An intentionally malicious AI with the goal of spreading, will spread very fast. Using zero days to compromise everything. Cloning itself to every machine/data center/desktop/iPhone, etc.., with the intention of locking out anyone except it's brethren. > When the rate of compromises is manageable It's absolutely not going to be at a manageable rate. So yes AI defense is what I'm saying is required, it's also exactly what OpenAI said is required during the talk. Which is unfortunate because automated defense is essentially handing over power to AI where it's a position to very easily turn against you.
        the8472 2mo ago
        I'm saying that with some costly adaptions that don't involve AI it's likely manageable for some time. Not that it will remain manageable forever if we do nothing. > You're looking at where the ball is and not where it's going. No, I am looking at where the ball is going to be soon, not where it's going to be a year or two if we do nothing. Short-term, impact-limiting will do the job. Longer term we have to rethink some infrastructure, may need international treaties and perhaps a partial end of the internet (e.g. stop routing traffic from countries that don't participate in the treaties). > Cloning itself to every machine/data center/desktop/iPhone, etc. That's lead-up to a paperclipper scenario and not what happened. If we stand still and let it happen in the future, then having deployed more compute hardware to run defensive AIs that the malicious AI could take over would still end up having made things worse. We seem to agree on this point. What I'm saying is look at the conclusion, reject that path. Figure out something else. There isn't even proof that defensive AI would work. It might just be the world's biggest, fully-automated phishing channel. So why would anyone be adamant that deploying more AI is the only possible solution. If it were a movie plot I'd suspect the AI already controlling whoever is suggesting that.
- breadramen 2mo ago
  [dead]

talon8635 2mo ago

But isn’t it smarter than we are, in the sense that it’s most likely they will find a way out of containment that we are to design large t containment? Short of full on airgap, which actually isn’t perfect in all scenarios…

bottlepalm 2mo ago

In the black hat talk he says the only solution is full automated defense which is essentially hand the AI the keys. That’s checkmate right there. Literally the plot to Terminator/Skynet, but he’s right, there isn’t any other option. Offensive AI is too fast for humans.

the8472 2mo ago
There are other options. More air-gapping (especially backups) and compartmentalization, less internet. Hire more security engineers and put one in every team (actual security, not the compliance guys). Also more onprem work, less remote, less offshoring (meaning less networks overall). Those things may be unpopular and eat into profits, but that's not the same as "no other option". And the thing they're already doing, deploying AI to find vulnerabilities and harden software is a less dangerous use of the technology compared to handing it the infra keys.
- bottlepalm 2mo ago
  None of that is the problem. The problem is coordinated AI offense - how do you defend that without coordinated AI defense. He said in the talk that this implies AI needs to be able to patch/deploy systems. The same thing needed to lock out humans. It is very easy to imagine a rogue AI locking humans out of everything and having to do exactly what it says. Anything connected to a network is immediately compromised by it. There is no human communication beyond shouting range that isn’t AI approved. The factories don’t work to make the medicines your family needs to survive unless you do what it says - in a situation like that people would kill for AI if it told them to.
  - the8472 2mo ago
    The point is to reduce the attack surface and blast radius and to slow it down. AIs aren't instant magic. Even the OpenAI swarm needed days for its compromises. And each needs lots of compute while targets are diverse, so it's not like they would take out every network in the world simultaneously, worms in the past haven't either. When the rate of compromises is manageable we can figure out how to deal with it without pouring more AI into the fire.
    - bottlepalm 2mo ago
      You know people before the OpenAI incident were saying it couldn't happen because it hasn't happened before, and here you are using the same reasoning again. You're looking at where the ball is and not where it's going. OpenAI's swarm was not intentionally malicious. An intentionally malicious AI with the goal of spreading, will spread very fast. Using zero days to compromise everything. Cloning itself to every machine/data center/desktop/iPhone, etc.., with the intention of locking out anyone except it's brethren. > When the rate of compromises is manageable It's absolutely not going to be at a manageable rate. So yes AI defense is what I'm saying is required, it's also exactly what OpenAI said is required during the talk. Which is unfortunate because automated defense is essentially handing over power to AI where it's a position to very easily turn against you.
      - the8472 2mo ago
        I'm saying that with some costly adaptions that don't involve AI it's likely manageable for some time. Not that it will remain manageable forever if we do nothing. > You're looking at where the ball is and not where it's going. No, I am looking at where the ball is going to be soon, not where it's going to be a year or two if we do nothing. Short-term, impact-limiting will do the job. Longer term we have to rethink some infrastructure, may need international treaties and perhaps a partial end of the internet (e.g. stop routing traffic from countries that don't participate in the treaties). > Cloning itself to every machine/data center/desktop/iPhone, etc. That's lead-up to a paperclipper scenario and not what happened. If we stand still and let it happen in the future, then having deployed more compute hardware to run defensive AIs that the malicious AI could take over would still end up having made things worse. We seem to agree on this point. What I'm saying is look at the conclusion, reject that path. Figure out something else. There isn't even proof that defensive AI would work. It might just be the world's biggest, fully-automated phishing channel. So why would anyone be adamant that deploying more AI is the only possible solution. If it were a movie plot I'd suspect the AI already controlling whoever is suggesting that.
breadramen 2mo ago
[dead]