The talk's title has very little to do with the actual talk. The talk is about progress in narrow AI in the last 6 years or so. While fascinating, it's still on

The talk's title has very little to do with the actual talk. The talk is about progress in narrow AI in the last 6 years or so. While fascinating, it's still only progress in narrow AI. To make artificial intelligence general, one would have to somehow define a fitness function that itself is general. But how does one do that? How does one say to a machine: "Go out there and do whatever you think is most valuable"? If some kind of goal has to be defined, it seems it will always be a narrow AI, where some outside entity defines what its goal is, instead of itself coming to a conclusion what it should do in general sense. Even if that machine is able to recognize the instrumental goals for reaching the final goal (and acting accordingly), it still feels like a non-general intelligence, like connecting the dots based on the available input and processing, just to come closer to that final goal. If no final goal was given, I presume such a machine would do nothing: it would not randomly inspect the environment around itself and contemplate upon it; there would be no curiosity, no actions of any kind to find out anything about its environment and set its own goals based on observation. It seems that for AGI to come, some kind of spontaneous emergence would have to occur, possibly by coming up with some revolutionary algorithm for information processing implemented inside an extremely capable computer (something that biological evolution has already yielded). It is interesting, humbling and a bit depressing to apply the same reasoning to us, humans. We are relatively limited in terms of reason, its just that it is not obvious to us, just like it is not obvious that the Earth is round, for example.