4 ms·

Some context on Goertzel that wouldn't show up in the wiki article. Goertzel's belief is/was that one couldn't achieve general / human-levels intelligence by sc

Some context on Goertzel that wouldn't show up in the wiki article. Goertzel's belief is/was that one couldn't achieve general / human-levels intelligence by scaling up regular old neural network ideas. He, and some others, believed that while traditional AI techniques were good within confined use cases, writing an intelligence that could apply itself to problem domains far outside of its training set required an explicitly different set of algorithms at the general/meta level. E.g. I imagine that Goertzel would say that GPT-3 isn't a general intelligence. It's the human engineer that made GPT-3, or which builds applications applying GPT-3 to different domains that is the general intelligence. Yes GPT-3 is generally applicable, but who does the hard work of actually figuring out how to apply it to new setups? THAT is general intelligence, in Goertzel's view. Back to the timeline, in 2002 Goertzel wanted to focus not on machine learning, which the field of AI was and still is mostly focused on, but rather designing "something" which (in current day terms) knew how to adapt meta-parameters to a particular training data set, and knew how to transfer/apply trained networks to new problem domains. An artificial AI engineer, as it were. So Goertzel coined/repurposed the term Artificial General Intelligence to mean not just a human-level AI, but rather a meta-AI program that was able to create, on its own, new human-level narrow AI using more traditional techniques. They created a AGI conference to gather researchers interested in these ideas: http://agi-conference.org http://agi-conference.org Generally speaking AI researchers who believe that scaling up traditional neural-network architectures is sufficient to create a general/human-level intelligence don't use the term AGI. The kind of folks who go to the AGI conference and study large-architecture AIs aiming for an explicit human-level, meta cognition engine do use the term AGI, and use it a lot. LessWrong folks may not hang around AGI conferences, but they do fall in the latter group. The MIRI institute was traditionally fully devoted to studying AGI architectures of their own trade-secret design, not narrow AI techniques (although that has changed a bit with Paul Christiano's work). So if you use AGI without qualification, my prior is that you're hailing from the latter camp of AGI meta-architecture researchers or the LessWrong-influenced existential risk crowd. In either case it is worth pointing out that you are getting a tinted view of AI risk that is different from what, say, researchers at DeepMind or Facebook largely believe.

drdeca 5y ago
I suppose I should be more up-front about "yes, I am rather influenced by the LessWrong/AlignmentForum views on AI risk". Also, it sounds like I may be over-estimating how commonly held some of their views are? (Though, I generally consider the chances of AGI actually ever being developed, to be much lower than they do, but for primarily religious reasons (as a reason that it wouldn't, not couldn't, happen), and am interested in their work.) My impression of the people on AlignmentForum wasn't that they were confident that "just scale up neural nets and use the right training procedure" couldn't work, but yes that there is a qualitative difference between what we have, and something which is as general-purpose as a human (not necessarily as competent, but capable of reasoning about the world and it being embedded in it, etc.). Personally, I find it hard to imagine that there could not be a qualitative difference in at least the training procedure. But also, (while it might not be the most efficient way to achieve it, even by many orders of magnitude, or it could be, idk) I find it hard to imagine that it would be impossible for a neural net, with the appropriate architecture and training procedure, to form one. (At the least in a trivial sense of "you can represent whatever known computation you want with something that could at least somewhat-reasonably be called a neural net") ____ Regarding that history of Goertzel's views and such, that is interesting, thank you! Yes, I suppose falling in with the LW/Miri/AlignmentForum camp and Goertzel, I would agree that gpt3 isn't a general intelligence. I've been perhaps so mired in that background that I, find it hard to imagine disagreeing with that, so perhaps I need to seek out some other views? So, I guess maybe I need to better understand the views of researchers at deepmind and facebook, etc. I'm aware that DeepMind had their paper about training agents in diverse environments, which I thought was cool. This to me, and things like it, would seem to me to be qualitatively different (in e.g. how it is trained) than e.g. gpt3 , even though both are neural nets.
- garmaine 5y ago
  I lost ten years of my life to worrying about AI risk. I try not to be bitter about that as the rationality techniques I learned along the way have helped me immensely. But I do try to caution others who appear to be falling down the same rabbit hole. It is the AI x-risk crowd that is non-mainstream, so there isn't really an easily accessible collection of opposing viewpoints, because few have bothered to write up their views. The default/null position is that AI, like nearly all other technologies ever, will have a gradual impact on society and will remain subject to human controls. And that's what most people believe, so researchers who don't buy into the x-risk scenario simply dismiss it as they do all crackpot ideas and move on with their lives. (It should be said that Goertzel has written up his response though: http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html http://multiverseaccordingtoben.blogspot.com/2010/10/singula...) "GPT-3 is AGI" is a bit of a strawman, sorry. I don't think anyone believes that. But GPT-3 is a clear demonstration that networks can be trained on todays hardware to have at least enough encoded information as a human mind, and encoded in such a way as to be generically useful. But GPT-3 is not an agent, obviously. In the mind of its creators however, it ought to be relatively simple (read: a few million dollars and a couple of years) to attach the GPT-3 knowledgebase to the kind of reinforcement learning based agent architecture that DeepMind and OpenAI are independently developing. Do that and you would have an agent-y general, human-level (or better) intelligence. Am I worried about that? No. Nor are most researchers in the field, for a variety of reasons. Some say it will be so fragile that it will require a symbiotic relationship with its human handlers, and unable to realistically act against us. Others say that its neural network architecture and human training data will make it behave similar enough to its creators in the ways which matter. Others note that initial projects will be so bug-ridden as to be unusable for word domination until decades down the line, and AI defenses will naturally be developed at a similar pace. So AI might become a geopolitical arms race, but not any different than nuclear weapons in that regard. Those are all viable arguments I've heard, and I'm sure there are others as well. Notably for AI x-risk to be a thing, all of the above arguments have to be flawed. If even one of them is true, AI will not take over the world. So we don't even have to delve deep into each of them to assess that the likely outcome is that AI does not kill us all. Edit: Also I think you're right that the AlignmentForum crowd doesn't think that "AGI" requires old-fashioned AI techniques instead of neural networks, but rather that they find those techniques easier to analyze and they're taking a maths-heavy approach to friendly-AI. Even Goertzel thinks that a neural-network approach could work; he just thinks (or thought?) that it would take too long to develop. Now it's probably the sunk cost fallacy and pride that keeps him developing OpenCog as an AGI architecture instead, even though the most basic AI-gym bots have surpassed OpenCog in capability.

drdeca 5y ago

I suppose I should be more up-front about "yes, I am rather influenced by the LessWrong/AlignmentForum views on AI risk". Also, it sounds like I may be over-estimating how commonly held some of their views are? (Though, I generally consider the chances of AGI actually ever being developed, to be much lower than they do, but for primarily religious reasons (as a reason that it wouldn't, not couldn't, happen), and am interested in their work.) My impression of the people on AlignmentForum wasn't that they were confident that "just scale up neural nets and use the right training procedure" couldn't work, but yes that there is a qualitative difference between what we have, and something which is as general-purpose as a human (not necessarily as competent, but capable of reasoning about the world and it being embedded in it, etc.). Personally, I find it hard to imagine that there could not be a qualitative difference in at least the training procedure. But also, (while it might not be the most efficient way to achieve it, even by many orders of magnitude, or it could be, idk) I find it hard to imagine that it would be impossible for a neural net, with the appropriate architecture and training procedure, to form one. (At the least in a trivial sense of "you can represent whatever known computation you want with something that could at least somewhat-reasonably be called a neural net") ____ Regarding that history of Goertzel's views and such, that is interesting, thank you! Yes, I suppose falling in with the LW/Miri/AlignmentForum camp and Goertzel, I would agree that gpt3 isn't a general intelligence. I've been perhaps so mired in that background that I, find it hard to imagine disagreeing with that, so perhaps I need to seek out some other views? So, I guess maybe I need to better understand the views of researchers at deepmind and facebook, etc. I'm aware that DeepMind had their paper about training agents in diverse environments, which I thought was cool. This to me, and things like it, would seem to me to be qualitatively different (in e.g. how it is trained) than e.g. gpt3 , even though both are neural nets.

garmaine 5y ago
I lost ten years of my life to worrying about AI risk. I try not to be bitter about that as the rationality techniques I learned along the way have helped me immensely. But I do try to caution others who appear to be falling down the same rabbit hole. It is the AI x-risk crowd that is non-mainstream, so there isn't really an easily accessible collection of opposing viewpoints, because few have bothered to write up their views. The default/null position is that AI, like nearly all other technologies ever, will have a gradual impact on society and will remain subject to human controls. And that's what most people believe, so researchers who don't buy into the x-risk scenario simply dismiss it as they do all crackpot ideas and move on with their lives. (It should be said that Goertzel has written up his response though: http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html http://multiverseaccordingtoben.blogspot.com/2010/10/singula...) "GPT-3 is AGI" is a bit of a strawman, sorry. I don't think anyone believes that. But GPT-3 is a clear demonstration that networks can be trained on todays hardware to have at least enough encoded information as a human mind, and encoded in such a way as to be generically useful. But GPT-3 is not an agent, obviously. In the mind of its creators however, it ought to be relatively simple (read: a few million dollars and a couple of years) to attach the GPT-3 knowledgebase to the kind of reinforcement learning based agent architecture that DeepMind and OpenAI are independently developing. Do that and you would have an agent-y general, human-level (or better) intelligence. Am I worried about that? No. Nor are most researchers in the field, for a variety of reasons. Some say it will be so fragile that it will require a symbiotic relationship with its human handlers, and unable to realistically act against us. Others say that its neural network architecture and human training data will make it behave similar enough to its creators in the ways which matter. Others note that initial projects will be so bug-ridden as to be unusable for word domination until decades down the line, and AI defenses will naturally be developed at a similar pace. So AI might become a geopolitical arms race, but not any different than nuclear weapons in that regard. Those are all viable arguments I've heard, and I'm sure there are others as well. Notably for AI x-risk to be a thing, all of the above arguments have to be flawed. If even one of them is true, AI will not take over the world. So we don't even have to delve deep into each of them to assess that the likely outcome is that AI does not kill us all. Edit: Also I think you're right that the AlignmentForum crowd doesn't think that "AGI" requires old-fashioned AI techniques instead of neural networks, but rather that they find those techniques easier to analyze and they're taking a maths-heavy approach to friendly-AI. Even Goertzel thinks that a neural-network approach could work; he just thinks (or thought?) that it would take too long to develop. Now it's probably the sunk cost fallacy and pride that keeps him developing OpenCog as an AGI architecture instead, even though the most basic AI-gym bots have surpassed OpenCog in capability.