9 ms·
I gave this talk a while ago to a small group of attendees. It was not recorded (I saw some ask below). It's based on a document I wrote a while ago called "You
by karpathy 9y ago
I gave this talk a while ago to a small group of attendees. It was not recorded (I saw some ask below). It's based on a document I wrote a while ago called "You suck at writing AI" (never published). The basic argument was that people are comically inadequate at writing complex code. You can't write the code to detect a cat in an image and the correct thing to do is to give up, write down an objective that measures the desiderata and pay with compute to search a function space for solutions. In the same vain, the idea of writing an AGI and all of its cognitive machinery is preposterous and the correct thing to do is to give up, think about the objective and search the program space for solutions. Unfortunately, the mindset of decomposition by function (see Brooks ref), which has worked so well for us in so many areas of scientific inquiry, is just about the most misleading mindset when it comes to AGI.
- olegkikin 9y agoFortunately you don't have to write an AGI yourself. We have a very powerful tool called evolution that can do all the heavy lifting for us, we just have to set up the environment and the goals. I'm pretty sure we could create an AGI, given enough computational power and time, we're basically hardware-limited.
- deleted 9y ago[deleted]
- goatlover 9y agoNature had a planet and several billion years. Of course the goal wasn't AGI, it was survival, and horseshoe crabs have done a pretty good job of that. So have beatles, with their large number of species. How would you select for only AGI? It would be like selecting for only the greatest eyes (Mantis shrimp), but doing it over the entire tree of life. You still need a way to narrow down to the best eyes.
- lgas 9y agoSurvival is not a goal, it's just the score keeping system.
- akvadrako 9y agoWhat, oh wise one, is the goal?
- LukaEn 9y agoNothing. Evoultion came about randomly, not by intent, so it has no goal.
- lgas 9y ago^
- goatlover 9y agoBut if the goal is to use evolution as model for generating AGI, then the problem is narrowing down all the surviving species to the ones that result in AGI, as opposed to a thousand other successful traits.
- akvadrako 9y agoThat's pretty meaningless - in that case the word goal doesn't have a meaning because everything is just as random as evolution, which isn't very random but whatever.
- visarga 9y agoEvolution is not random. It requires self replication in order to transmit and evolve genes. Everything else is much more random.
- posterboy 9y agoReduction of entropy. Which seems quite the opposite of growth. However effectively, say energy per volume, it allows for growth.
- kobeya 9y agoI think that's what the person you are replying to is arguing for...
- CuriouslyC 9y agoPlot twist: figuring out that objective function may prove as intractable as the original problem! We'll need an objective function writing objective function, and then it's turtles all the way down.
- bad_alloc 9y agoUnless it were possible for each objective function to converge to a next one that slightly extends the current one. However, what would the starting point be? Is this mathematically definable? What does the (im)possibility to define such a function tell us about AI?
- p1esk 9y agoMake them hierarchical. Starting point is to survive.
- visarga 9y ago> However, what would the starting point be? The starting place is simulation. We use games (and other kinds of sims) to learn to act intelligently in a virtual environment. In such a place we can define many tasks and a learning curriculum.
- mannigfaltig 9y agoThe main objective function in nature is very simple though: maximize the number of copies of your genes. Such an objective enforced in a resource-constrained multi-agent world (as suggested in the slides) could really lead to quite complex sub-objectives which may as well lead to general intelligence. For example, if each agent can process information and perform work, it follows that individuals that have better cooperative abilities also have a larger reproductive success. Cooperation is, however, extremely complex: It requires communication, identification of other individuals, establishment of trust, early detection of betrayal etc. The necessity for modeling the actions of other agents alone provides plenty of correction signals toward general intelligence because modelling other agents is such a difficult task.
- p1esk 9y agoIt's unfortunate that AI researchers pay so little attention to the brain (a single slide in your talk). We need more ML people evaluating ideas from neuroscience or from such initiatives as Numenta or Spaun.
- mycat 9y agoRemeber Koniku? (http://koniku.io/ http://koniku.io/) Biology and Biotech are different beasts. Saw their Ted talk, looks promising, but seems they delayed a lot.
- visarga 9y agoDemis Hassabis (Deep Mind) published recently a manifesto about the importance of neuroscience for AI. Neuroscience-Inspired Artificial Intelligence http://www.cell.com/neuron/abstract/S0896-6273(17)30509-3 http://www.cell.com/neuron/abstract/S0896-6273(17)30509-3
- paulsutter 9y agoPractical example, neural architecture search to discover convnets that require less computation, https://arxiv.org/abs/1707.07012 https://arxiv.org/abs/1707.07012 "Learning Transferable Architectures for Scalable Image Recognition"
- eb3c90 9y agoI'd be interested in what you think of my approach elsewhere in the replies. In some ways it is a search through program space. But then I think we as intelligences do a guided search through program space as we develop. It seems important to use what information you can to guide you e.g. culture and copying other people who have already been navigating the space of programs to be.
- russdill 9y agoIs there a video of your giving the talk? Or at least audio?
- deleted 9y ago[deleted]
- idlewords 9y agoWould you be willing to write this talk out in longer form? I realize it's a lot of work, but the slides are fantastic and even an intermediate textual version that just fills in some of the explanatory gaps would be fascinating.
- jayunit 9y agoAny thoughts on what role novelty search might play? ("Why Greatness Cannot Be Planned: The Myth of the Objective" by Kenneth Stanley)
- d_burfoot 9y agoThere are at least two huge advantages of the decomposition by function approach compared to the black box solution space search approach: - The former can be debugged. If your image analysis system embarrassingly tags black people as gorillas, you can go in and find the bug and fix it. That's not so easy if you're using a black box model. - The black box approach is tremendously enervating. You code up a neural network architecture, launch a bunch of cloud GPU instances, and start training. If the results are bad... you try a new architecture. In the function decomposition world, you can actually use your knowledge and understanding of the system as an engineer to figure out what went wrong and why.
- LukeB42 9y agoThe objective function for humans seems to be "Perfect the Human Soul" which can have conflicting optima in different contexts. Obviously no other mammal optimizes for it, either. See any slice of human culture or human activity ever for reference. Thanks for putting this slideshow together.
- malmsteen 9y agoNah it's just "survive and replicate". like every other species. Except that it's harder for us than other species, in particular, because a small human child is super fragile (more than like a giraffe baby) so you need a lot of care, and a lot of intelligence to do that care.
- LukeB42 9y agoNah I maintain "Survive and replicate" is a sub-problem in the larger "Perfect ones' Soul" objective function.
- visarga 9y agoWhat is a "soul"? I can define "survive and replicate" for you, on the other hand. <rant>This exemplifies the issue I have with current day philosophy. It's too blissfully unaware of the discoveries in AI. While they redefine consciousness the 1000th time, the AI researchers make "Reinforcement Learning Agents" that play Go, drive cars, paint, draw and can take a pizza order from you. Philosophers, get more concrete.</>
- marmaduke 9y agoAnyway you might publish the original writing? I have been skeptical of the very notion of AGI ever since hearing it described as a sort of computational or scientific panacea; it would be nice to have more discussion on the topic.