4 ms·

I suppose I should be more up-front about "yes, I am rather influenced by the LessWrong/AlignmentForum views on AI risk". Also, it sounds like I may be over-est

by drdeca 5y ago

I suppose I should be more up-front about "yes, I am rather influenced by the LessWrong/AlignmentForum views on AI risk". Also, it sounds like I may be over-estimating how commonly held some of their views are? (Though, I generally consider the chances of AGI actually ever being developed, to be much lower than they do, but for primarily religious reasons (as a reason that it wouldn't, not couldn't, happen), and am interested in their work.) My impression of the people on AlignmentForum wasn't that they were confident that "just scale up neural nets and use the right training procedure" couldn't work, but yes that there is a qualitative difference between what we have, and something which is as general-purpose as a human (not necessarily as competent, but capable of reasoning about the world and it being embedded in it, etc.). Personally, I find it hard to imagine that there could not be a qualitative difference in at least the training procedure. But also, (while it might not be the most efficient way to achieve it, even by many orders of magnitude, or it could be, idk) I find it hard to imagine that it would be impossible for a neural net, with the appropriate architecture and training procedure, to form one. (At the least in a trivial sense of "you can represent whatever known computation you want with something that could at least somewhat-reasonably be called a neural net") ____ Regarding that history of Goertzel's views and such, that is interesting, thank you! Yes, I suppose falling in with the LW/Miri/AlignmentForum camp and Goertzel, I would agree that gpt3 isn't a general intelligence. I've been perhaps so mired in that background that I, find it hard to imagine disagreeing with that, so perhaps I need to seek out some other views? So, I guess maybe I need to better understand the views of researchers at deepmind and facebook, etc. I'm aware that DeepMind had their paper about training agents in diverse environments, which I thought was cool. This to me, and things like it, would seem to me to be qualitatively different (in e.g. how it is trained) than e.g. gpt3 , even though both are neural nets.

garmaine 5y ago
I lost ten years of my life to worrying about AI risk. I try not to be bitter about that as the rationality techniques I learned along the way have helped me immensely. But I do try to caution others who appear to be falling down the same rabbit hole. It is the AI x-risk crowd that is non-mainstream, so there isn't really an easily accessible collection of opposing viewpoints, because few have bothered to write up their views. The default/null position is that AI, like nearly all other technologies ever, will have a gradual impact on society and will remain subject to human controls. And that's what most people believe, so researchers who don't buy into the x-risk scenario simply dismiss it as they do all crackpot ideas and move on with their lives. (It should be said that Goertzel has written up his response though: http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html http://multiverseaccordingtoben.blogspot.com/2010/10/singula...) "GPT-3 is AGI" is a bit of a strawman, sorry. I don't think anyone believes that. But GPT-3 is a clear demonstration that networks can be trained on todays hardware to have at least enough encoded information as a human mind, and encoded in such a way as to be generically useful. But GPT-3 is not an agent, obviously. In the mind of its creators however, it ought to be relatively simple (read: a few million dollars and a couple of years) to attach the GPT-3 knowledgebase to the kind of reinforcement learning based agent architecture that DeepMind and OpenAI are independently developing. Do that and you would have an agent-y general, human-level (or better) intelligence. Am I worried about that? No. Nor are most researchers in the field, for a variety of reasons. Some say it will be so fragile that it will require a symbiotic relationship with its human handlers, and unable to realistically act against us. Others say that its neural network architecture and human training data will make it behave similar enough to its creators in the ways which matter. Others note that initial projects will be so bug-ridden as to be unusable for word domination until decades down the line, and AI defenses will naturally be developed at a similar pace. So AI might become a geopolitical arms race, but not any different than nuclear weapons in that regard. Those are all viable arguments I've heard, and I'm sure there are others as well. Notably for AI x-risk to be a thing, all of the above arguments have to be flawed. If even one of them is true, AI will not take over the world. So we don't even have to delve deep into each of them to assess that the likely outcome is that AI does not kill us all. Edit: Also I think you're right that the AlignmentForum crowd doesn't think that "AGI" requires old-fashioned AI techniques instead of neural networks, but rather that they find those techniques easier to analyze and they're taking a maths-heavy approach to friendly-AI. Even Goertzel thinks that a neural-network approach could work; he just thinks (or thought?) that it would take too long to develop. Now it's probably the sunk cost fallacy and pride that keeps him developing OpenCog as an AGI architecture instead, even though the most basic AI-gym bots have surpassed OpenCog in capability.

by drdeca 5y ago

garmaine 5y ago

I lost ten years of my life to worrying about AI risk. I try not to be bitter about that as the rationality techniques I learned along the way have helped me immensely. But I do try to caution others who appear to be falling down the same rabbit hole. It is the AI x-risk crowd that is non-mainstream, so there isn't really an easily accessible collection of opposing viewpoints, because few have bothered to write up their views. The default/null position is that AI, like nearly all other technologies ever, will have a gradual impact on society and will remain subject to human controls. And that's what most people believe, so researchers who don't buy into the x-risk scenario simply dismiss it as they do all crackpot ideas and move on with their lives. (It should be said that Goertzel has written up his response though: http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html http://multiverseaccordingtoben.blogspot.com/2010/10/singula...) "GPT-3 is AGI" is a bit of a strawman, sorry. I don't think anyone believes that. But GPT-3 is a clear demonstration that networks can be trained on todays hardware to have at least enough encoded information as a human mind, and encoded in such a way as to be generically useful. But GPT-3 is not an agent, obviously. In the mind of its creators however, it ought to be relatively simple (read: a few million dollars and a couple of years) to attach the GPT-3 knowledgebase to the kind of reinforcement learning based agent architecture that DeepMind and OpenAI are independently developing. Do that and you would have an agent-y general, human-level (or better) intelligence. Am I worried about that? No. Nor are most researchers in the field, for a variety of reasons. Some say it will be so fragile that it will require a symbiotic relationship with its human handlers, and unable to realistically act against us. Others say that its neural network architecture and human training data will make it behave similar enough to its creators in the ways which matter. Others note that initial projects will be so bug-ridden as to be unusable for word domination until decades down the line, and AI defenses will naturally be developed at a similar pace. So AI might become a geopolitical arms race, but not any different than nuclear weapons in that regard. Those are all viable arguments I've heard, and I'm sure there are others as well. Notably for AI x-risk to be a thing, all of the above arguments have to be flawed. If even one of them is true, AI will not take over the world. So we don't even have to delve deep into each of them to assess that the likely outcome is that AI does not kill us all. Edit: Also I think you're right that the AlignmentForum crowd doesn't think that "AGI" requires old-fashioned AI techniques instead of neural networks, but rather that they find those techniques easier to analyze and they're taking a maths-heavy approach to friendly-AI. Even Goertzel thinks that a neural-network approach could work; he just thinks (or thought?) that it would take too long to develop. Now it's probably the sunk cost fallacy and pride that keeps him developing OpenCog as an AGI architecture instead, even though the most basic AI-gym bots have surpassed OpenCog in capability.