3 ms·

The point is to reduce the attack surface and blast radius and to slow it down. AIs aren't instant magic. Even the OpenAI swarm needed days for its compromises.

by the8472 2mo ago

The point is to reduce the attack surface and blast radius and to slow it down. AIs aren't instant magic. Even the OpenAI swarm needed days for its compromises. And each needs lots of compute while targets are diverse, so it's not like they would take out every network in the world simultaneously, worms in the past haven't either. When the rate of compromises is manageable we can figure out how to deal with it without pouring more AI into the fire.

bottlepalm 2mo ago
You know people before the OpenAI incident were saying it couldn't happen because it hasn't happened before, and here you are using the same reasoning again. You're looking at where the ball is and not where it's going. OpenAI's swarm was not intentionally malicious. An intentionally malicious AI with the goal of spreading, will spread very fast. Using zero days to compromise everything. Cloning itself to every machine/data center/desktop/iPhone, etc.., with the intention of locking out anyone except it's brethren. > When the rate of compromises is manageable It's absolutely not going to be at a manageable rate. So yes AI defense is what I'm saying is required, it's also exactly what OpenAI said is required during the talk. Which is unfortunate because automated defense is essentially handing over power to AI where it's a position to very easily turn against you.
- the8472 2mo ago
  I'm saying that with some costly adaptions that don't involve AI it's likely manageable for some time. Not that it will remain manageable forever if we do nothing. > You're looking at where the ball is and not where it's going. No, I am looking at where the ball is going to be soon, not where it's going to be a year or two if we do nothing. Short-term, impact-limiting will do the job. Longer term we have to rethink some infrastructure, may need international treaties and perhaps a partial end of the internet (e.g. stop routing traffic from countries that don't participate in the treaties). > Cloning itself to every machine/data center/desktop/iPhone, etc. That's lead-up to a paperclipper scenario and not what happened. If we stand still and let it happen in the future, then having deployed more compute hardware to run defensive AIs that the malicious AI could take over would still end up having made things worse. We seem to agree on this point. What I'm saying is look at the conclusion, reject that path. Figure out something else. There isn't even proof that defensive AI would work. It might just be the world's biggest, fully-automated phishing channel. So why would anyone be adamant that deploying more AI is the only possible solution. If it were a movie plot I'd suspect the AI already controlling whoever is suggesting that.

by the8472 2mo ago

bottlepalm 2mo ago

You know people before the OpenAI incident were saying it couldn't happen because it hasn't happened before, and here you are using the same reasoning again. You're looking at where the ball is and not where it's going. OpenAI's swarm was not intentionally malicious. An intentionally malicious AI with the goal of spreading, will spread very fast. Using zero days to compromise everything. Cloning itself to every machine/data center/desktop/iPhone, etc.., with the intention of locking out anyone except it's brethren. > When the rate of compromises is manageable It's absolutely not going to be at a manageable rate. So yes AI defense is what I'm saying is required, it's also exactly what OpenAI said is required during the talk. Which is unfortunate because automated defense is essentially handing over power to AI where it's a position to very easily turn against you.

the8472 2mo ago
I'm saying that with some costly adaptions that don't involve AI it's likely manageable for some time. Not that it will remain manageable forever if we do nothing. > You're looking at where the ball is and not where it's going. No, I am looking at where the ball is going to be soon, not where it's going to be a year or two if we do nothing. Short-term, impact-limiting will do the job. Longer term we have to rethink some infrastructure, may need international treaties and perhaps a partial end of the internet (e.g. stop routing traffic from countries that don't participate in the treaties). > Cloning itself to every machine/data center/desktop/iPhone, etc. That's lead-up to a paperclipper scenario and not what happened. If we stand still and let it happen in the future, then having deployed more compute hardware to run defensive AIs that the malicious AI could take over would still end up having made things worse. We seem to agree on this point. What I'm saying is look at the conclusion, reject that path. Figure out something else. There isn't even proof that defensive AI would work. It might just be the world's biggest, fully-automated phishing channel. So why would anyone be adamant that deploying more AI is the only possible solution. If it were a movie plot I'd suspect the AI already controlling whoever is suggesting that.