3 ms·

The entire sandbox thing is foolish. Obviously it won’t be sandboxed anyway because of what you’ve pointed out. But any sandbox we design won’t be good enough.

by seaorg 5y ago

The entire sandbox thing is foolish. Obviously it won’t be sandboxed anyway because of what you’ve pointed out. But any sandbox we design won’t be good enough. GTP is not supposed to exist according to computer science and machine learning experts in 2017. They were all wrong. Same with deep dream, styleGAN. And it will be true with the next thing. The mental gymnastics refers to the fact that people have cognitive dissonance about GTP. They have amnesia about the decades of stagnant progress in text generation. All anyone can do is point out that it’s definitely not sentient and it’s not AGI so what’s the big deal? All of a sudden the goal posts have been moved… this is the single biggest quantum leap in text generation ever, a mind-boggling level of lucidity with grammar, punctuation and more all without ever being given a single deliberate instruction from a human being. It is objectively amazing. Objectively. And people have no reaction. Because they are not emotionally ready for it. It’s cognitive dissonance, mental gymnastics, whatever you want to call it.

helen___keller 5y ago
> But any sandbox we design won’t be good enough. This isn't really true. To bring back your analogy of searching the space of all programs - suppose we wanted to simulate every turing machine of at most N states for some sufficiently large N. One of these turing machines is going to encode an AGI. Nonetheless, it is still just an encoded turing machine being simulated by a turing machine simulator. No matter what great intelligence is encoded by those states, it's still a state machine following the same rules as every other turing machine in existence. It's a fixed set of states following a fixed set of rules on an unbounded-length tape of memory. The same is true for an AGI in a computer program in a modern computer. A complex modern neural nets are still just a program that takes a giant array of numbers and feeds it through a network of transformations repeatedly, yielding a new array of numbers. No matter what brilliant information that array of numbers encodes, the array of numbers is still just a bunch of bits somewhere in RAM. It's data, not execution. It can't invoke the IPC stack unless *we* program functionality in that interprets the data and executes functionality based on it. It's like the brain in a vat[0] thought experiment. A super-intelligent brain in a vat can't escape the lab unless the lab researchers give the brain control over something physical. A super-intelligent brain can't even know that it's a brain in the vat unless it is told. Why then would you expect a digital brain in a simulated environment to even know that it is what it is, much less how to perfectly execute a bug in a simulator (which might provably not exist[1]) to send a copy of its source code to another computer and end up replicating and distributing itself? The simple answer is: Sandboxing an AI is significantly easier than you seem to think. When an AI takes over the world, it's going to be because someone handed the AI root access to a big tech data center so it can optimize operations; not because the AI discovered the perfect sequence of zero-day bugs to overcome all of humanity's best cybersecurity efforts. [0] https://en.wikipedia.org/wiki/Brain_in_a_vat https://en.wikipedia.org/wiki/Brain_in_a_vat [1] https://en.wikipedia.org/wiki/Formal_verification https://en.wikipedia.org/wiki/Formal_verification
- seaorg 5y ago
  Yes, programs are Turing machines that use memory. I’m a little insulted. No method of containment is good enough for something like that. But we agree that sandboxing as a basis to claim that AGI won’t be a danger is totally unsound.

by seaorg 5y ago

helen___keller 5y ago

> But any sandbox we design won’t be good enough. This isn't really true. To bring back your analogy of searching the space of all programs - suppose we wanted to simulate every turing machine of at most N states for some sufficiently large N. One of these turing machines is going to encode an AGI. Nonetheless, it is still just an encoded turing machine being simulated by a turing machine simulator. No matter what great intelligence is encoded by those states, it's still a state machine following the same rules as every other turing machine in existence. It's a fixed set of states following a fixed set of rules on an unbounded-length tape of memory. The same is true for an AGI in a computer program in a modern computer. A complex modern neural nets are still just a program that takes a giant array of numbers and feeds it through a network of transformations repeatedly, yielding a new array of numbers. No matter what brilliant information that array of numbers encodes, the array of numbers is still just a bunch of bits somewhere in RAM. It's data, not execution. It can't invoke the IPC stack unless *we* program functionality in that interprets the data and executes functionality based on it. It's like the brain in a vat[0] thought experiment. A super-intelligent brain in a vat can't escape the lab unless the lab researchers give the brain control over something physical. A super-intelligent brain can't even know that it's a brain in the vat unless it is told. Why then would you expect a digital brain in a simulated environment to even know that it is what it is, much less how to perfectly execute a bug in a simulator (which might provably not exist[1]) to send a copy of its source code to another computer and end up replicating and distributing itself? The simple answer is: Sandboxing an AI is significantly easier than you seem to think. When an AI takes over the world, it's going to be because someone handed the AI root access to a big tech data center so it can optimize operations; not because the AI discovered the perfect sequence of zero-day bugs to overcome all of humanity's best cybersecurity efforts. [0] https://en.wikipedia.org/wiki/Brain_in_a_vat https://en.wikipedia.org/wiki/Brain_in_a_vat [1] https://en.wikipedia.org/wiki/Formal_verification https://en.wikipedia.org/wiki/Formal_verification

seaorg 5y ago
Yes, programs are Turing machines that use memory. I’m a little insulted. No method of containment is good enough for something like that. But we agree that sandboxing as a basis to claim that AGI won’t be a danger is totally unsound.