4 ms·

The layman and the AI expert have both written off the possibility of creating AGI for the entirety of this latest AI spring. In the past decade I have basicall

by seaorg 5y ago

The layman and the AI expert have both written off the possibility of creating AGI for the entirety of this latest AI spring. In the past decade I have basically never encountered anyone who thought that AGI was going to happen in our lifetimes or even anyone who believed that it would be a problem if it did. One time I discussed AGI with a good friend. And I gently pressured him to play through the scenario of the advent of AGI. And he made a guilty confession to me. I could tell he was embarrassed to share his opinion. He said “can’t we just unplug it?” This is a microcosm of the entire issue. It’s something a child might say. For global warming, can’t we just turn on the air conditioning? No, we can’t just unplug it. Here is how you can understand what is happening. An AI model is just a program. A program that is written by another program. Consider a list of programs that contains every possible program. It is infinite. Make it finite by limiting the size of the programs based on some practical consideration. It’s still a very long list. What is in this list? Pong. StyleGAN. AGI? Almost certainly. How small might AGI be? It probably appears more than once on the list — how many are there? But let’s ask the real question here and boil this down. How many items on the list are something we will regret having discovered? A great, great many. Every day, people are exploring the list. They are using primitive methods to sift through all these programs and find the ones that have interesting behavior. The process of program discovery is automated with things called “training algorithms.” As our computers get faster and capital allocation grows larger we mine this list more and more quickly. The problem is that we keep finding things that surprise us. And that is the core and substance of the entire issue. We have demonstrated to ourselves over and over again that our own intuition about the contents of this list is completely wrong. Despite the mind-boggling level of mental gymnastics performed in the wake of GTP-3, enough to constitute a three ring circus, the fact of the matter is that GTP-3 and soon to be 4 is not supposed to exist according to every single AI expert. Compute is only getting cheaper and capital is only getting bigger and the list is only getting shorter and it’s not going to stop surprising everyone including the “experts.” Just because we don’t know how to make something we will truly regret doesn’t mean we aren’t capable of creating it.

helen___keller 5y ago
> He said “can’t we just unplug it?” This is a microcosm of the entire issue. It’s something a child might say. For global warming, can’t we just turn on the air conditioning? No, we can’t just unplug it. If it were running as a sandboxed application inside some kind of runtime environment with only access to print output text and read input text, as you might expect a GPT-n program to run, certainly you could unplug it. Humanity could be considered God's AGI, but if God doesn't give us the APIs to interact with heaven, it follows that we can't kill God. Of course, humans are much more greedy, so I'm sure our creations will have full access to any APIs needed to run our factories, advertise products on the internet, and maybe direct our military weapons too :) Oops! > Despite the mind-boggling level of mental gymnastics performed in the wake of GTP-3, enough to constitute a three ring circus, the fact of the matter is that GTP-3 and soon to be 4 is not supposed to exist according to every single AI expert Could you elaborate? What mental gymnastics, and it is not supposed to exist according to who exactly?
- seaorg 5y ago
  The entire sandbox thing is foolish. Obviously it won’t be sandboxed anyway because of what you’ve pointed out. But any sandbox we design won’t be good enough. GTP is not supposed to exist according to computer science and machine learning experts in 2017. They were all wrong. Same with deep dream, styleGAN. And it will be true with the next thing. The mental gymnastics refers to the fact that people have cognitive dissonance about GTP. They have amnesia about the decades of stagnant progress in text generation. All anyone can do is point out that it’s definitely not sentient and it’s not AGI so what’s the big deal? All of a sudden the goal posts have been moved… this is the single biggest quantum leap in text generation ever, a mind-boggling level of lucidity with grammar, punctuation and more all without ever being given a single deliberate instruction from a human being. It is objectively amazing. Objectively. And people have no reaction. Because they are not emotionally ready for it. It’s cognitive dissonance, mental gymnastics, whatever you want to call it.
  - helen___keller 5y ago
    > But any sandbox we design won’t be good enough. This isn't really true. To bring back your analogy of searching the space of all programs - suppose we wanted to simulate every turing machine of at most N states for some sufficiently large N. One of these turing machines is going to encode an AGI. Nonetheless, it is still just an encoded turing machine being simulated by a turing machine simulator. No matter what great intelligence is encoded by those states, it's still a state machine following the same rules as every other turing machine in existence. It's a fixed set of states following a fixed set of rules on an unbounded-length tape of memory. The same is true for an AGI in a computer program in a modern computer. A complex modern neural nets are still just a program that takes a giant array of numbers and feeds it through a network of transformations repeatedly, yielding a new array of numbers. No matter what brilliant information that array of numbers encodes, the array of numbers is still just a bunch of bits somewhere in RAM. It's data, not execution. It can't invoke the IPC stack unless *we* program functionality in that interprets the data and executes functionality based on it. It's like the brain in a vat[0] thought experiment. A super-intelligent brain in a vat can't escape the lab unless the lab researchers give the brain control over something physical. A super-intelligent brain can't even know that it's a brain in the vat unless it is told. Why then would you expect a digital brain in a simulated environment to even know that it is what it is, much less how to perfectly execute a bug in a simulator (which might provably not exist[1]) to send a copy of its source code to another computer and end up replicating and distributing itself? The simple answer is: Sandboxing an AI is significantly easier than you seem to think. When an AI takes over the world, it's going to be because someone handed the AI root access to a big tech data center so it can optimize operations; not because the AI discovered the perfect sequence of zero-day bugs to overcome all of humanity's best cybersecurity efforts. [0] https://en.wikipedia.org/wiki/Brain_in_a_vat https://en.wikipedia.org/wiki/Brain_in_a_vat [1] https://en.wikipedia.org/wiki/Formal_verification https://en.wikipedia.org/wiki/Formal_verification

by seaorg 5y ago

helen___keller 5y ago

> He said “can’t we just unplug it?” This is a microcosm of the entire issue. It’s something a child might say. For global warming, can’t we just turn on the air conditioning? No, we can’t just unplug it. If it were running as a sandboxed application inside some kind of runtime environment with only access to print output text and read input text, as you might expect a GPT-n program to run, certainly you could unplug it. Humanity could be considered God's AGI, but if God doesn't give us the APIs to interact with heaven, it follows that we can't kill God. Of course, humans are much more greedy, so I'm sure our creations will have full access to any APIs needed to run our factories, advertise products on the internet, and maybe direct our military weapons too :) Oops! > Despite the mind-boggling level of mental gymnastics performed in the wake of GTP-3, enough to constitute a three ring circus, the fact of the matter is that GTP-3 and soon to be 4 is not supposed to exist according to every single AI expert Could you elaborate? What mental gymnastics, and it is not supposed to exist according to who exactly?

seaorg 5y ago
The entire sandbox thing is foolish. Obviously it won’t be sandboxed anyway because of what you’ve pointed out. But any sandbox we design won’t be good enough. GTP is not supposed to exist according to computer science and machine learning experts in 2017. They were all wrong. Same with deep dream, styleGAN. And it will be true with the next thing. The mental gymnastics refers to the fact that people have cognitive dissonance about GTP. They have amnesia about the decades of stagnant progress in text generation. All anyone can do is point out that it’s definitely not sentient and it’s not AGI so what’s the big deal? All of a sudden the goal posts have been moved… this is the single biggest quantum leap in text generation ever, a mind-boggling level of lucidity with grammar, punctuation and more all without ever being given a single deliberate instruction from a human being. It is objectively amazing. Objectively. And people have no reaction. Because they are not emotionally ready for it. It’s cognitive dissonance, mental gymnastics, whatever you want to call it.
- helen___keller 5y ago
  > But any sandbox we design won’t be good enough. This isn't really true. To bring back your analogy of searching the space of all programs - suppose we wanted to simulate every turing machine of at most N states for some sufficiently large N. One of these turing machines is going to encode an AGI. Nonetheless, it is still just an encoded turing machine being simulated by a turing machine simulator. No matter what great intelligence is encoded by those states, it's still a state machine following the same rules as every other turing machine in existence. It's a fixed set of states following a fixed set of rules on an unbounded-length tape of memory. The same is true for an AGI in a computer program in a modern computer. A complex modern neural nets are still just a program that takes a giant array of numbers and feeds it through a network of transformations repeatedly, yielding a new array of numbers. No matter what brilliant information that array of numbers encodes, the array of numbers is still just a bunch of bits somewhere in RAM. It's data, not execution. It can't invoke the IPC stack unless *we* program functionality in that interprets the data and executes functionality based on it. It's like the brain in a vat[0] thought experiment. A super-intelligent brain in a vat can't escape the lab unless the lab researchers give the brain control over something physical. A super-intelligent brain can't even know that it's a brain in the vat unless it is told. Why then would you expect a digital brain in a simulated environment to even know that it is what it is, much less how to perfectly execute a bug in a simulator (which might provably not exist[1]) to send a copy of its source code to another computer and end up replicating and distributing itself? The simple answer is: Sandboxing an AI is significantly easier than you seem to think. When an AI takes over the world, it's going to be because someone handed the AI root access to a big tech data center so it can optimize operations; not because the AI discovered the perfect sequence of zero-day bugs to overcome all of humanity's best cybersecurity efforts. [0] https://en.wikipedia.org/wiki/Brain_in_a_vat https://en.wikipedia.org/wiki/Brain_in_a_vat [1] https://en.wikipedia.org/wiki/Formal_verification https://en.wikipedia.org/wiki/Formal_verification