4 ms·

Maybe I'm missing something here. First of all, I know a lot of the popular perception of artificial "general intelligence" is overly simplistic - I don't beli

Maybe I'm missing something here. First of all, I know a lot of the popular perception of artificial "general intelligence" is overly simplistic - I don't believe in the nerd rapture, and I agree with a lot of what was written in the three articles tweeted by Chollet that have been linked in this thread. And yet, I still don't see how AI is not a plausible existential threat. Unless you believe that some magical business is going on in the human mind that isn't subject to the normal laws of physics, then I don't see how you can believe that there's anything our brains can do that another machine can't. Even if no cognitive skill can be increased to infinity, we have no good reason to believe that our brains represent the maximum performance of all possible cognitive modes. That's an appeal to the discredited idea that evolution has a "ladder", upon which we stand at the apex as the finished product. Natural selection doesn't optimize for intelligence, and it is not "finished". So, if our brains are machines (albeit highly complex ones that we only partially understand), and if they probably don't represent the maximum potential performance of cognition, then how can we say with any confidence that it is not possible to create another machine with higher cognitive performance across all (or nearly all) modes of cognition? And if that is possible, how can we say with confidence that such machines could pose no existential threat to us? Sure, maybe they won't, or maybe we'll never figure out how to build them. But how is it implausible? How is it something to laugh out of the room? Furthermore, AI need not surpass us in all modes of cognition in order to be an existential threat. As AI gets better at accomplishing a wide variety of tasks, it becomes an ever more powerful lever for those who own it. The near-term threat from AI is socioeconomic: the replacement of vast numbers of jobs with AI/robotics controlled by a small number of people who receive all the profits of their "labor". It doesn't take much imagination to see how thie could be, at the least, an existential threat to our current society if it is not addressed with adequate forethought. All in all, I just do not see how AI is not an existential threat worth thinking about and sinking some money into researching how we can make it safer. The tired old argument about the need to spend those resources on more urgent matters doesn't hold water. There are seven billion of us. We can specialize - indeed, it's arguably our greatest strength! We can - and must! - devote resources and talent to a great many urgent issues, such as poverty, conflict, disease, and illiteracy. But I think we would be very unwise not to put a little of our wealth and time into researching how best to mitigate the long-term threats that don't seem urgent yet and might not even come to pass. If we don't, then chances are someday one of them will in fact come to pass and we'll wish we had worked on it sooner.

jph00 9y ago
I agree with everything you said. However, I don't agree it's what should be hogging the money and interest right now. The impact of AI on employment (and therefore societal stability) is a much more pressing problem. As is the misuse of AI by people in many areas. If society can get through these issues, then we may get to the point where the existential threat of superhuman AI is of most immediate concern. Either way, we're not arguing that this issue deserves no time and attention at all. But currently it's the first (and often only) thing I'm asked about when I give talks about the future implications of AI, and it's receiving huge amounts of funding. Elon Musk, when he had the opportunity of addressing some of America's most powerful people, elected to spend his precious time discussing this issue, rather than anything else.
- bryananderson 9y ago
  I definitely agree that it shouldn't hog all the money and interest - I just don't think that it's hogging a disproportionate share right now. Musk and Altman's OpenAI is a drop in the bucket in the universe of philanthropy. In the AI field, the bulk of work at companies and research at univesities is not going toward existential risk mitigation. I think the "If society can get through these issues, then..." perspective is fundamentally flawed. We will always, always have urgent problems. Guanranteed. We need to work hard on them. But we also need to work a little bit on the big things that are not of most immediate concern. I think a great analogy for this is climate change. If people in the early Industrial Revolution knew what we know now about greenhouse gases, they wouldn't have been inclined to worry. The amount they were putting into the atmosphere just was not significant, and even if it rose sharply, it would be many generations before there might be an issue. And yet, if people had put some focus on long-term threats, we might be in less dire straits now. I definitely sympathize with your experience as an AI educator being constantly peppered with hypotheticals about superintelligence. I love your fast.ai course and I am sure you'd rather talk about teaching AI than about something that may or may not be an issue decades from now. But that doesn't mean the issue doesn't deserve more funding. Musk's topic choice may seem repetitive to us, as we are in the tech world and read about/discuss this topic all the time. But a bunch of older politician types probably have not even been exposed to these ideas of existential risk, and as they hold a lot of power, it's not a bad thing to educate them on. For what it's worth, Musk also spent a lot of time discussing nearer-term implications like job displacement.

jph00 9y ago

I agree with everything you said. However, I don't agree it's what should be hogging the money and interest right now. The impact of AI on employment (and therefore societal stability) is a much more pressing problem. As is the misuse of AI by people in many areas. If society can get through these issues, then we may get to the point where the existential threat of superhuman AI is of most immediate concern. Either way, we're not arguing that this issue deserves no time and attention at all. But currently it's the first (and often only) thing I'm asked about when I give talks about the future implications of AI, and it's receiving huge amounts of funding. Elon Musk, when he had the opportunity of addressing some of America's most powerful people, elected to spend his precious time discussing this issue, rather than anything else.

bryananderson 9y ago
I definitely agree that it shouldn't hog all the money and interest - I just don't think that it's hogging a disproportionate share right now. Musk and Altman's OpenAI is a drop in the bucket in the universe of philanthropy. In the AI field, the bulk of work at companies and research at univesities is not going toward existential risk mitigation. I think the "If society can get through these issues, then..." perspective is fundamentally flawed. We will always, always have urgent problems. Guanranteed. We need to work hard on them. But we also need to work a little bit on the big things that are not of most immediate concern. I think a great analogy for this is climate change. If people in the early Industrial Revolution knew what we know now about greenhouse gases, they wouldn't have been inclined to worry. The amount they were putting into the atmosphere just was not significant, and even if it rose sharply, it would be many generations before there might be an issue. And yet, if people had put some focus on long-term threats, we might be in less dire straits now. I definitely sympathize with your experience as an AI educator being constantly peppered with hypotheticals about superintelligence. I love your fast.ai course and I am sure you'd rather talk about teaching AI than about something that may or may not be an issue decades from now. But that doesn't mean the issue doesn't deserve more funding. Musk's topic choice may seem repetitive to us, as we are in the tech world and read about/discuss this topic all the time. But a bunch of older politician types probably have not even been exposed to these ideas of existential risk, and as they hold a lot of power, it's not a bad thing to educate them on. For what it's worth, Musk also spent a lot of time discussing nearer-term implications like job displacement.