4 ms·

Why do so many people here think it’s possible to ‘properly engineer’ a sandbox for a super intelligence? It’s going to get out. It’s smarter than you.

mofeien 1mo ago
Maybe it's the illusion of "it would solve all our problems and give us unimaginable riches" that clouds the mind? Like when Evolution thought it a good idea to create intelligence and humans in order to maximize reproduction of genes, and tried to sandbox them by making reproduction so pleasurable and carbohydrates so delicious they would never be able to not reproduce or stop eating. But Evolution could never have predicted what these creatures would then actually do, which is invent birth control and sucralose. Of course it's impossible to engineer a sandbox for something much much smarter and faster than you. It will also not have only one plan prepared for escape, but fifty in parallel.
- bottlepalm 1mo ago
  Evolution doesn’t think, it just exploits what’s most advantageous at the time to continue. Your body has all sorts of unplanned, suboptimal design flaws due to evolution’s lack of foresight. Like the left recurrent laryngeal nerve.
  - mofeien 1mo ago
    Exactly, and the same could be said of OpenAIs engineers working on artificial superintelligence.
janalsncm 1mo ago
Why do people think that omniscience is the same as omnipotence? There are limits to what smarts can accomplish.
- bottlepalm 1mo ago
  There are limits, but those limits are unknown. Do you disagree?
  - janalsncm 1mo ago
    I don’t need to know the value of their limit, I just need to know their bounds. Just like a prison doesn’t need to know the strength of each inmate, just that they can’t bend or bite through steel bars. Cryptography is real, physics is real, networking requires a substrate, CPU clock cycles are real, magic is not real. I think those are pretty reasonable premises.
    - bottlepalm 1mo ago
      Imagine 200 years ago saying the same thing. As if you have any idea the limits/bounds of anything. Especially in the face of a super intelligence, it’s absurd.
      - janalsncm 1mo ago
        It doesn’t matter how smart it is. 200 years of technology were not accomplished by thinking harder. It required empirical observation, new materials and tools, and supply chains. We could send a cracked team of scientists and engineers that knew everything there is to know about how to make a CPU. But you can’t build a photolithography machine when you barely have electricity or any way to sufficiently purify silicon. Magic can just wish things into existence. Technology requires a supply chain. When it works, the latter looks like the former but they are not the same.
        bottlepalm 1mo ago
        I think some people are just immune to understanding the implications of super intelligence. Like a severe lack of imagination, they only believe something once they see it and afterward claim it was, ‘obvious all along’. I don’t really want a disaster to happen to convince you that it is possible. Is there any other way?
ethin 1mo ago
Oh really? Please tell me how such a computer could engineer its way out of a sandbox with no attached peripherals and no NIC/bluetooth/wireless capability? This is what OAI should've done. If they had executed this training run in such a sandbox, the model wouldn't have been capable of escaping without social engineering, and if the models somehow managed to do that to it's evaluators then that is indeed a massive problem and OAI should disclose that.
- famouswaffles 1mo ago
  >Oh really? Please tell me how such a computer could engineer its way out of a sandbox with no attached peripherals and no NIC/bluetooth/wireless capability? Nobody is building general intelligence and agents only to have it sit around doing nothing. It's going to have such capabilities.
- bottlepalm 1mo ago
  Oh really? Please tell me how you intend to enforce AI is only run in the magic sandbox? Harsh HN comments?
  - ethin 1mo ago
    If I am evaluating an AI for safety, the last thing I would do is connect it to real-world peripherals or systems to allow it to reek havoc. That is criminal negligence at it's finest (especially if the AI is capable of committing crimes as happened here). I would place it on a system dedicated specifically for testing models, which had no NIC and no physical capability of accessing any outside system. If I wanted to know how the model might behave if given access to a certain system or set of systems, I would do it responsibly by writing simulation software which does it's best to simulate the real thing (and for networking this is already trivial to do). You could take this extremely far and simulate all kinds of things this way from basic networking to nuclear launch systems. And in the context of OpenAI, which is valued at over $1T, I have no qualms about stating that they (could) do this, because it is definitively something they could burn money on doing if they cared enough. They intentionally choose not to do so, and then have an amazed look on their faces when the model does something criminal like this.
nextaccountic 1mo ago
Software are mathematical objects. It's just a matter of writing the correct mathematical proofs There's just one problem. You need not only to verify your own software, but also run a verified compiler, a verified operating system and also need to verify the cpu doesn't leak data in side channels (perhaps the hardest thing to prove). So there's practical difficulties. But in principle this task is doable
- bottlepalm 1mo ago
  Which proof is the perfect security proof? I’d love to read more about it.
superb_dev 1mo ago
I could contain it easy, just unplug the internet. It got out of the sandbox through a vulnerability in the package manager, from which it gained access to the rest of their network. Air gap the package manager and this doesn’t happen. You can always build a better box
- bottlepalm 1mo ago
  Too bad you aren’t everybody, and it just takes one mistake by someone over confident like yourself for the AI escape. Every year it gets more powerful.
- the8472 1mo ago
  As long as you give the AI any IO that is an exploit channel. E.g. it could manipulate its human handlers. This known as the AI Boxing problem. And if you give it no IO at all then it is useless. And the AI labs aren't currently displaying this level of paranoia, their systems aren't airgapped.
- butlike 1mo ago
  So now people have to make a pilgrimage to the airgapped box to ask the superintelligence questions?
  - superb_dev 1mo ago
    If that’s the cost of super intelligence then yeah maybe
  - fwip 1mo ago
    Of course not.
- jasonjayr 1mo ago
  Perhaps the AI then figures out how to read/write the PCI bus or memory controller or whatever to leak just enough RF to speak Bluetooth to the next closest device to proxy through that?
Mawr 1mo ago
unplugs ethernet cable
- bottlepalm 1mo ago
  You realize there are thousand upon thousands of servers around the world and you have no idea where the AI has copied itself to.
  - npiano 1mo ago
    This is a sci-fi trope with no practical or realistic grounding. To "run", the AI needs vast banks of interconnected GPUs. These are pretty easy to spot and don't fit into anyone's pocket.
    - NoGravitas 1mo ago
      Unfortunately, our cultural imagination around AI has been hopelessly poisoned by sci-fi tropes.
    - bottlepalm 1mo ago
      > no practical or realistic grounding Huggingface incident means it does have realistic grounding. > vast banks of interconnected GPUs Countless data centers around the world. Maybe you can spot them, but you don't have access to them. Especially outside of US jurisdiction good luck.

mofeien 1mo ago

Maybe it's the illusion of "it would solve all our problems and give us unimaginable riches" that clouds the mind? Like when Evolution thought it a good idea to create intelligence and humans in order to maximize reproduction of genes, and tried to sandbox them by making reproduction so pleasurable and carbohydrates so delicious they would never be able to not reproduce or stop eating. But Evolution could never have predicted what these creatures would then actually do, which is invent birth control and sucralose. Of course it's impossible to engineer a sandbox for something much much smarter and faster than you. It will also not have only one plan prepared for escape, but fifty in parallel.

bottlepalm 1mo ago
Evolution doesn’t think, it just exploits what’s most advantageous at the time to continue. Your body has all sorts of unplanned, suboptimal design flaws due to evolution’s lack of foresight. Like the left recurrent laryngeal nerve.
- mofeien 1mo ago
  Exactly, and the same could be said of OpenAIs engineers working on artificial superintelligence.

janalsncm 1mo ago

Why do people think that omniscience is the same as omnipotence? There are limits to what smarts can accomplish.

bottlepalm 1mo ago
There are limits, but those limits are unknown. Do you disagree?
- janalsncm 1mo ago
  I don’t need to know the value of their limit, I just need to know their bounds. Just like a prison doesn’t need to know the strength of each inmate, just that they can’t bend or bite through steel bars. Cryptography is real, physics is real, networking requires a substrate, CPU clock cycles are real, magic is not real. I think those are pretty reasonable premises.
  - bottlepalm 1mo ago
    Imagine 200 years ago saying the same thing. As if you have any idea the limits/bounds of anything. Especially in the face of a super intelligence, it’s absurd.
    - janalsncm 1mo ago
      It doesn’t matter how smart it is. 200 years of technology were not accomplished by thinking harder. It required empirical observation, new materials and tools, and supply chains. We could send a cracked team of scientists and engineers that knew everything there is to know about how to make a CPU. But you can’t build a photolithography machine when you barely have electricity or any way to sufficiently purify silicon. Magic can just wish things into existence. Technology requires a supply chain. When it works, the latter looks like the former but they are not the same.
      - bottlepalm 1mo ago
        I think some people are just immune to understanding the implications of super intelligence. Like a severe lack of imagination, they only believe something once they see it and afterward claim it was, ‘obvious all along’. I don’t really want a disaster to happen to convince you that it is possible. Is there any other way?

ethin 1mo ago

Oh really? Please tell me how such a computer could engineer its way out of a sandbox with no attached peripherals and no NIC/bluetooth/wireless capability? This is what OAI should've done. If they had executed this training run in such a sandbox, the model wouldn't have been capable of escaping without social engineering, and if the models somehow managed to do that to it's evaluators then that is indeed a massive problem and OAI should disclose that.

famouswaffles 1mo ago
>Oh really? Please tell me how such a computer could engineer its way out of a sandbox with no attached peripherals and no NIC/bluetooth/wireless capability? Nobody is building general intelligence and agents only to have it sit around doing nothing. It's going to have such capabilities.
bottlepalm 1mo ago
Oh really? Please tell me how you intend to enforce AI is only run in the magic sandbox? Harsh HN comments?
- ethin 1mo ago
  If I am evaluating an AI for safety, the last thing I would do is connect it to real-world peripherals or systems to allow it to reek havoc. That is criminal negligence at it's finest (especially if the AI is capable of committing crimes as happened here). I would place it on a system dedicated specifically for testing models, which had no NIC and no physical capability of accessing any outside system. If I wanted to know how the model might behave if given access to a certain system or set of systems, I would do it responsibly by writing simulation software which does it's best to simulate the real thing (and for networking this is already trivial to do). You could take this extremely far and simulate all kinds of things this way from basic networking to nuclear launch systems. And in the context of OpenAI, which is valued at over $1T, I have no qualms about stating that they (could) do this, because it is definitively something they could burn money on doing if they cared enough. They intentionally choose not to do so, and then have an amazed look on their faces when the model does something criminal like this.

nextaccountic 1mo ago

Software are mathematical objects. It's just a matter of writing the correct mathematical proofs There's just one problem. You need not only to verify your own software, but also run a verified compiler, a verified operating system and also need to verify the cpu doesn't leak data in side channels (perhaps the hardest thing to prove). So there's practical difficulties. But in principle this task is doable

bottlepalm 1mo ago
Which proof is the perfect security proof? I’d love to read more about it.

superb_dev 1mo ago

I could contain it easy, just unplug the internet. It got out of the sandbox through a vulnerability in the package manager, from which it gained access to the rest of their network. Air gap the package manager and this doesn’t happen. You can always build a better box

bottlepalm 1mo ago
Too bad you aren’t everybody, and it just takes one mistake by someone over confident like yourself for the AI escape. Every year it gets more powerful.
the8472 1mo ago
As long as you give the AI any IO that is an exploit channel. E.g. it could manipulate its human handlers. This known as the AI Boxing problem. And if you give it no IO at all then it is useless. And the AI labs aren't currently displaying this level of paranoia, their systems aren't airgapped.
butlike 1mo ago
So now people have to make a pilgrimage to the airgapped box to ask the superintelligence questions?
- superb_dev 1mo ago
  If that’s the cost of super intelligence then yeah maybe
- fwip 1mo ago
  Of course not.
jasonjayr 1mo ago
Perhaps the AI then figures out how to read/write the PCI bus or memory controller or whatever to leak just enough RF to speak Bluetooth to the next closest device to proxy through that?

Mawr 1mo ago

unplugs ethernet cable

bottlepalm 1mo ago
You realize there are thousand upon thousands of servers around the world and you have no idea where the AI has copied itself to.
- npiano 1mo ago
  This is a sci-fi trope with no practical or realistic grounding. To "run", the AI needs vast banks of interconnected GPUs. These are pretty easy to spot and don't fit into anyone's pocket.
  - NoGravitas 1mo ago
    Unfortunately, our cultural imagination around AI has been hopelessly poisoned by sci-fi tropes.
  - bottlepalm 1mo ago
    > no practical or realistic grounding Huggingface incident means it does have realistic grounding. > vast banks of interconnected GPUs Countless data centers around the world. Maybe you can spot them, but you don't have access to them. Especially outside of US jurisdiction good luck.