7 ms·

Artificial intelligence risk research

43 points by BenjaminTodd 11y ago

andreyf 11y ago
The possibility of human-level artificial intelligence poses significant risks to society [...] No. It simply does not outside of the realm of science fiction. Understanding HGI will help us understand enough about ourselves and our motivations and ethics that it won't be an issue. Humans augmented by machine learning systems, on the other hand, are here now and a wholly different question. Institutions empowered by AI research have a power / person ratio unlike any of those of the past.
- Moshe_Silnorin 11y ago
  >It simply does not outside of the realm of science fiction. Why?
  - andreyf 11y ago
    In short, because human intelligence evolved in a different environment than AI. We are the results of the need to survive and reproduce, and the majority of our "intelligence" is based on this foundation. Algorithms have no such darwinian pressures, and there is no point in developing AI replacements for them, since humans abound.
    - Moshe_Silnorin 11y ago
      I thought much the same once. This paper convinced me otherwise: https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf https://selfawaresystems.files.wordpress.com/2008/01/ai_driv... I'd be interested in your take on it.
      - andreyf 11y ago
        Only skimmed it, but seems like my thoughts would be similar to manish_gill's above [1]: the assumptions are rooted in pop science fantasy. More specifically, there are an infinite number of reasons for and against doing anything. Once we understand how humans weigh these reasons and choose between them and model them in an HGI system, we will be able to give it the values and morals that its creators choose. 1. https://news.ycombinator.com/item?id=10724108 https://news.ycombinator.com/item?id=10724108
    - feral 11y ago
      You could make basically the same argument for why humans will never invent heavier-than-air flight - because avian flight 'evolved based on a need to survive and reproduce', and 'artificial flight doesn't have darwinian pressures'. But humans did make artificial flying machines. I don't mean to be unkind, but your argument doesn't have close to the level of certainly that we should bet our species on. No one knows where the roadblocks to AGI are, or what the timelines might be; but there is what seems like huge progress happening recently in an area which might eventually lead there. While no one knows the path, many intelligent people have thought about this without finding any theoretical roadblocks. Please don't publicly dismiss the concerns as 'science fiction' without a little more thought.
      - andreyf 11y ago
        I'm not saying that HGI is impossible, but that the "risk scenarios" are complete fiction. In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. This will change the questions of risk into something that's tangible and addressable.
        feral 11y ago
        On Ada Lovelace's 200 year birthday, we might remember her quote: "The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform." She was very smart, but she was wrong about that. >In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. You mean that it's impossible to make a human level general intelligence ('HGI', to use your term), without understanding those things? If so, I disagree with your assertion: it is not obvious that those things have to be done before a human level AGI could be made. It might be a good idea to do so, but its not clear that they are a necessary requirement. Some people would say this is why AGI is dangerous. Maybe there's a path to bootstrapping an AGI, where we build something from relatively simple bits we can understand, that then becomes very much more complex than we can understand. This happens all the time: for example, we get very complex classification behaviour from deep nets: [simple pieces] + [lots of data] doing something more complex than we have explicitly trained them to. If we were convinced that "In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior" was a necessary thing to build before we could build a HGI, people would be less worried. But its not; and that's partially why people are worried. In fact, some of the people working on AI risk could be said to be basically racing to understand these things before other folk build an AGI.
    - robbensinger 11y ago
      Stuart Russell (co-author of AI:MA, one of MIRI's research advisors) argues on http://edge.org/conversation/the-myth-of-ai#26015 http://edge.org/conversation/the-myth-of-ai#26015 that AI systems with "the ability to make high-quality decisions" (where "quality refers to the expected outcome utility of actions taken" and the utility function is represented in the system's programmed decision criteria) raises two problems: "1. The utility function may not be perfectly aligned with the values of the human race, which are (at best) very difficult to pin down. "2. Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task." The first of those is what Bostrom calls "perverse instantiation" and Dietterich and Horvitz call the "Sorcerer's Apprentice" problem (http://cacm.acm.org/magazines/2015/10/192386-rise-of-concerns-about-ai/fulltext http://cacm.acm.org/magazines/2015/10/192386-rise-of-concern...). The second of these is what Bostrom calls "convergent instrumental goals" and Omohundro calls "basic AI drives." The first of these seems like a fairly obvious problem, if we think AI systems will ever be trusted with making important decisions. Human goals are complicated, and even a superintelligent system that can easily learn about our goals won't necessarily acquire the goals thereby. So solving the AI problem doesn't get us a solution to the goal specification problem for free. The second of these also has some intuitive force; https://intelligence.org/?p=12234 https://intelligence.org/?p=12234 shows Omohundro's idea can be stated formally, so it's not purely sci-fi. Averting the "Sorcerer's Apprentice" problem in full generality would mean averting this problem, since we'd then simply be able to give AI systems the right goals and let them go wild. Absent that, if AI systems become much more cognitively capable than humans, we'll probably need to actively work on some approach that violates Omohundro's assumptions (and the assumptions of the formalism above). Bostrom and MIRI both talk about a lot of interesting ideas along these lines.
      - andreyf 11y ago
        What is "an AI system with the ability to make high-quality decisions"? Do automated derivative trading models count? Do systems which decide how much to bid on a RTB ad exchange count? The first problem is not new. We have a similar problem with some corporations, for example. "A sufficiently capable intelligent system" is as real as "sufficiently hostile aliens". It's hard to argue and reason about a fictional system with a assortment of properties picked by someone aiming to spreading fear.
- platz 11y ago
  I'm not sure that because you envision a different, more plausible scenario, that completely invalidates the original claim, which was only simply denied before sketching out the alternative. * I don't see how "Understanding HGI" prevents a "human-level artificial intelligence" from doing anything it pleases.
  - andreyf 11y ago
    My second claim is not there to invalidate the first, but rather to point out that there are two "AI problems", and that one is a lot closer to science fiction than the other. I've expounded on the denial: we will need to first figure out what motivates humans before we can replicate their levels of intelligence. So in order to solve HGI, we will need to understand HGI, which means we will be able to control HGI. We will understand personalities and values in a systematic way, and we will be able to choose the personalities and values of any one instance of HGI.
    - platz 11y ago
      > we will need to first figure out what motivates humans before we can replicate their levels of intelligence I would not assume this with certainty. There are many systems in nature that consist of simple pieces, when composed, yield enormous complexity. Replicating intelligences might be like black boxes which we don't understand but can generate, or grow, if you will, from simpler primitives. We might get lucky and discover the means to generate intelligences without understanding them. As an example, "Machine learning works spectacularly well, but mathematicians aren’t quite sure why." https://www.quantamagazine.org/20151203-big-datas-mathematical-mysteries/ https://www.quantamagazine.org/20151203-big-datas-mathematic...
      - andreyf 11y ago
        Look at where we are now with ML, and ask what we would need to do to "accidentally" create HGI. That title's wording is clickbait, not an argument. Mathematicians know exactly why neural networks work – because they contain a model of the data, and they try to develop ways to build (train) those models so that they do it better. The model may not have human concepts associated with it, but that doesn't mean it's somehow "out of control". In order to achieve HGI, one would have to train models of quite a few domains (the same way schools create models in people's minds), as well as tons of ethical and practical heuristics to weigh an infinite number of reasons for or against doing something.
- conceit 11y ago
  the problem probably is human-level intelligence is still stupid, so it's a risk either way
- jerf 11y ago
  "It simply does not outside of the realm of science fiction." If you could prove that, you'd put a lot of minds at ease. By "prove" I mean not necessarily a full mathematical proof, but certainly more than a few paragraphs of English text. I'm skeptical myself, seems like the more we learn about biology the more it is doing that we did not realize, but skepticism is at best the first step to proof.
  - andreyf 11y ago
    I think it will be obvious once HGI is sufficiently understood, in a way that will not need more than a paragraph, if that. The fear and idea of somehow creating an HGI without understanding how to control it came from The Terminator. It's pure fantasy.
    - robertk 11y ago
      Factually disputed. The concerns raised in this area originated in the 90s, conceived prior to release of the first Terminator film.
      - andreyf 11y ago
        The first Terminator was released in 1984. The idea existed before it, of course.
    - eli_gottlieb 11y ago
      Just because we will understand, in principle, how to control it, doesn't mean we'll engineer that control particularly well.
    - jerf 11y ago
      That doesn't even rise to the level of proof-by-disbelief; it's proof by hypothetical future disbelief. You've given no more reason to be confident in your hypothetical than the concerns about hypothetical self-improving AIs. You're failing to sell this argument to someone generally inclined to agree with you about the outcome.
- BenjaminTodd 11y ago
  The purpose of the profile isn't to argue a risk exists. We largely defer to the people we take to be experts on the issue, especially Nick Bostrom. We think he presents compelling arguments in Superintelligence, and although it's hard to say anything decisive in this area, if you think there's even modest uncertainty about whether AGI will be good or bad, it's worth doing more research into the risks. If you haven't read Bostrom's book yet, I'd really recommend it. http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/1501227742 http://www.amazon.com/Superintelligence-Dangers-Strategies-N...
  - manish_gill 11y ago
    I read this book. Got about 4 chapters in before I had to give up at the sheer ridiculousness of the whole thing. The problem with this entire line of reasoning is that at this point it is nothing more an a thought experiment. Many of the key underlying assumptions that are required for Artificial General Intelligence simply have not been realised, and while Weak AI is progressing at a strong rate, we are hardly anywhere close to a place where we should start worrying about all this stuff. It's about as likely as a meteor hitting the planet and wiping out the entire human life. Possible? Sure. Should I be panicking about this right now? Nah. The book is nonsense. I'll start paying attention when someone who has real experience in the field of AI research (and I'm not talking charlatans like Yudkowsky here), but someone like say Norvig come out and say it's a reasonable concern today.
    - BenjaminTodd 11y ago
      The expert consensus says 10% chance of human-level AI in 10 years: http://www.givewell.org/labs/causes/ai-risk/ai-timelines http://www.givewell.org/labs/causes/ai-risk/ai-timelines Many computer science professors have publicly said they think AI poses significant risks. There's a list here: http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-risk/ http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-ri... Also see this open letter, signed by hundreds of experts: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/
      - 11y ago
- ctogden 11y ago
  What is HGI? Google/Bing/Wikipedia disambiguation are no help.
  - andreyf 11y ago
    Human-level General Intelligence.
- danieltillett 11y ago
  Human level AI would have a profound impact on the value of human labor. If you don't consider everyone being unemployable an issue to society then I am not sure what would pose a significant risk. Most people that worry about this topic don't worry about human level AI, it is the vastly greater than human level AI that is the concern.
  - punee 11y ago
    How does human level AI make everyone unemployable?
    - danieltillett 11y ago
      If the AI is at human level then it can do all human jobs. Given that humans require a minimum income to survive, if the total per hour cost of AI is below this level then all jobs will be taken by AI (why employ an expensive human when you can get a cheaper AI). This is what happened to human computers (people, mostly women, who performed mathematical calculations) in the 1950s when electronic computers became cheaper and faster per calculation. All those jobs ceased to exist. There are some caveats here. The cost of AI labor needs to be below that of the human minimum. Given there is no effective limit to how much AI labor you can create, the likely result is the cost of capital will fall to near zero making AI labor near free. No human will be able to compete against this except in providing personal services to the owners of capital in fluff jobs. The more interesting thing is that with the cost of capital and labour at near zero we should get massive deflation to the point that it will be very cheap to provide all people with a very high basic income that will make the need for employment unnecessary. This would be a great world to live in, but it would have a profound effect on the society unlike anything since the rise of agriculture.
      - punee 11y ago
        "If the AI is at human level then it can do all human jobs." I don't see how that follows. How does a computer program do manual labor?
- throwaway808 11y ago
  Two examples outside of science fiction: - The horse population declined dramatically after their work abilities were exceeded on every dimension by machines - Homo Neanderthalensis disappeared after the appearance of Homo Sapiens. I'm not saying these are perfect analogies. But you are saying they're completely irrelevant. Why is that?
  - nairboon 11y ago
    Good old Homo Sapiens is already disappearing thanks to Homo Oeconomicus. I don't see a problem if we move on to Homo faber next.
- threatofrain 11y ago
  How are you so confident that the risks of machine learning are in the realm of science fiction? Machine learning, at least, has the plausible potential to permanently reduce demand on labor by capital owners. Undemanded laborers lose their ability to negotiate or strike as a power counterbalance to capital owners. At the very least, that future is too complicated to form such a confident opinion that the risks of machine learning are in the realm of science fiction.
  - andreyf 11y ago
    There are certainly changes happening (and more on the way!) as a result of a combination of Moore's Law and ML research. I don't think they are inherently significantly more risky any more than other technical advances as long as we have a chance to think about and understand them. We definitely need to think through the social implications of computers. I think it's important to separate those real developments from the movie-plot-threat of a super-AI escaping human control.
rhaps0dy 11y ago
Be sure to de-lurk if that's the case! I guess. A lot of people already did. http://lukemuehlhauser.com/if-youre-an-ai-safety-lurker-now-would-be-a-good-time-to-de-lurk http://lukemuehlhauser.com/if-youre-an-ai-safety-lurker-now-...

43 points by BenjaminTodd 11y ago

The possibility of human-level artificial intelligence poses significant risks to society [...] No. It simply does not outside of the realm of science fiction. Understanding HGI will help us understand enough about ourselves and our motivations and ethics that it won't be an issue. Humans augmented by machine learning systems, on the other hand, are here now and a wholly different question. Institutions empowered by AI research have a power / person ratio unlike any of those of the past.

Moshe_Silnorin 11y ago
>It simply does not outside of the realm of science fiction. Why?
- andreyf 11y ago
  In short, because human intelligence evolved in a different environment than AI. We are the results of the need to survive and reproduce, and the majority of our "intelligence" is based on this foundation. Algorithms have no such darwinian pressures, and there is no point in developing AI replacements for them, since humans abound.
  - Moshe_Silnorin 11y ago
    I thought much the same once. This paper convinced me otherwise: https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf https://selfawaresystems.files.wordpress.com/2008/01/ai_driv... I'd be interested in your take on it.
    - andreyf 11y ago
      Only skimmed it, but seems like my thoughts would be similar to manish_gill's above [1]: the assumptions are rooted in pop science fantasy. More specifically, there are an infinite number of reasons for and against doing anything. Once we understand how humans weigh these reasons and choose between them and model them in an HGI system, we will be able to give it the values and morals that its creators choose. 1. https://news.ycombinator.com/item?id=10724108 https://news.ycombinator.com/item?id=10724108
  - feral 11y ago
    You could make basically the same argument for why humans will never invent heavier-than-air flight - because avian flight 'evolved based on a need to survive and reproduce', and 'artificial flight doesn't have darwinian pressures'. But humans did make artificial flying machines. I don't mean to be unkind, but your argument doesn't have close to the level of certainly that we should bet our species on. No one knows where the roadblocks to AGI are, or what the timelines might be; but there is what seems like huge progress happening recently in an area which might eventually lead there. While no one knows the path, many intelligent people have thought about this without finding any theoretical roadblocks. Please don't publicly dismiss the concerns as 'science fiction' without a little more thought.
    - andreyf 11y ago
      I'm not saying that HGI is impossible, but that the "risk scenarios" are complete fiction. In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. This will change the questions of risk into something that's tangible and addressable.
      - feral 11y ago
        On Ada Lovelace's 200 year birthday, we might remember her quote: "The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform." She was very smart, but she was wrong about that. >In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. You mean that it's impossible to make a human level general intelligence ('HGI', to use your term), without understanding those things? If so, I disagree with your assertion: it is not obvious that those things have to be done before a human level AGI could be made. It might be a good idea to do so, but its not clear that they are a necessary requirement. Some people would say this is why AGI is dangerous. Maybe there's a path to bootstrapping an AGI, where we build something from relatively simple bits we can understand, that then becomes very much more complex than we can understand. This happens all the time: for example, we get very complex classification behaviour from deep nets: [simple pieces] + [lots of data] doing something more complex than we have explicitly trained them to. If we were convinced that "In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior" was a necessary thing to build before we could build a HGI, people would be less worried. But its not; and that's partially why people are worried. In fact, some of the people working on AI risk could be said to be basically racing to understand these things before other folk build an AGI.
  - robbensinger 11y ago
    Stuart Russell (co-author of AI:MA, one of MIRI's research advisors) argues on http://edge.org/conversation/the-myth-of-ai#26015 http://edge.org/conversation/the-myth-of-ai#26015 that AI systems with "the ability to make high-quality decisions" (where "quality refers to the expected outcome utility of actions taken" and the utility function is represented in the system's programmed decision criteria) raises two problems: "1. The utility function may not be perfectly aligned with the values of the human race, which are (at best) very difficult to pin down. "2. Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task." The first of those is what Bostrom calls "perverse instantiation" and Dietterich and Horvitz call the "Sorcerer's Apprentice" problem (http://cacm.acm.org/magazines/2015/10/192386-rise-of-concerns-about-ai/fulltext http://cacm.acm.org/magazines/2015/10/192386-rise-of-concern...). The second of these is what Bostrom calls "convergent instrumental goals" and Omohundro calls "basic AI drives." The first of these seems like a fairly obvious problem, if we think AI systems will ever be trusted with making important decisions. Human goals are complicated, and even a superintelligent system that can easily learn about our goals won't necessarily acquire the goals thereby. So solving the AI problem doesn't get us a solution to the goal specification problem for free. The second of these also has some intuitive force; https://intelligence.org/?p=12234 https://intelligence.org/?p=12234 shows Omohundro's idea can be stated formally, so it's not purely sci-fi. Averting the "Sorcerer's Apprentice" problem in full generality would mean averting this problem, since we'd then simply be able to give AI systems the right goals and let them go wild. Absent that, if AI systems become much more cognitively capable than humans, we'll probably need to actively work on some approach that violates Omohundro's assumptions (and the assumptions of the formalism above). Bostrom and MIRI both talk about a lot of interesting ideas along these lines.
    - andreyf 11y ago
      What is "an AI system with the ability to make high-quality decisions"? Do automated derivative trading models count? Do systems which decide how much to bid on a RTB ad exchange count? The first problem is not new. We have a similar problem with some corporations, for example. "A sufficiently capable intelligent system" is as real as "sufficiently hostile aliens". It's hard to argue and reason about a fictional system with a assortment of properties picked by someone aiming to spreading fear.
platz 11y ago
I'm not sure that because you envision a different, more plausible scenario, that completely invalidates the original claim, which was only simply denied before sketching out the alternative. * I don't see how "Understanding HGI" prevents a "human-level artificial intelligence" from doing anything it pleases.
- andreyf 11y ago
  My second claim is not there to invalidate the first, but rather to point out that there are two "AI problems", and that one is a lot closer to science fiction than the other. I've expounded on the denial: we will need to first figure out what motivates humans before we can replicate their levels of intelligence. So in order to solve HGI, we will need to understand HGI, which means we will be able to control HGI. We will understand personalities and values in a systematic way, and we will be able to choose the personalities and values of any one instance of HGI.
  - platz 11y ago
    > we will need to first figure out what motivates humans before we can replicate their levels of intelligence I would not assume this with certainty. There are many systems in nature that consist of simple pieces, when composed, yield enormous complexity. Replicating intelligences might be like black boxes which we don't understand but can generate, or grow, if you will, from simpler primitives. We might get lucky and discover the means to generate intelligences without understanding them. As an example, "Machine learning works spectacularly well, but mathematicians aren’t quite sure why." https://www.quantamagazine.org/20151203-big-datas-mathematical-mysteries/ https://www.quantamagazine.org/20151203-big-datas-mathematic...
    - andreyf 11y ago
      Look at where we are now with ML, and ask what we would need to do to "accidentally" create HGI. That title's wording is clickbait, not an argument. Mathematicians know exactly why neural networks work – because they contain a model of the data, and they try to develop ways to build (train) those models so that they do it better. The model may not have human concepts associated with it, but that doesn't mean it's somehow "out of control". In order to achieve HGI, one would have to train models of quite a few domains (the same way schools create models in people's minds), as well as tons of ethical and practical heuristics to weigh an infinite number of reasons for or against doing something.
conceit 11y ago
the problem probably is human-level intelligence is still stupid, so it's a risk either way
jerf 11y ago
"It simply does not outside of the realm of science fiction." If you could prove that, you'd put a lot of minds at ease. By "prove" I mean not necessarily a full mathematical proof, but certainly more than a few paragraphs of English text. I'm skeptical myself, seems like the more we learn about biology the more it is doing that we did not realize, but skepticism is at best the first step to proof.
- andreyf 11y ago
  I think it will be obvious once HGI is sufficiently understood, in a way that will not need more than a paragraph, if that. The fear and idea of somehow creating an HGI without understanding how to control it came from The Terminator. It's pure fantasy.
  - robertk 11y ago
    Factually disputed. The concerns raised in this area originated in the 90s, conceived prior to release of the first Terminator film.
    - andreyf 11y ago
      The first Terminator was released in 1984. The idea existed before it, of course.
  - eli_gottlieb 11y ago
    Just because we will understand, in principle, how to control it, doesn't mean we'll engineer that control particularly well.
  - jerf 11y ago
    That doesn't even rise to the level of proof-by-disbelief; it's proof by hypothetical future disbelief. You've given no more reason to be confident in your hypothetical than the concerns about hypothetical self-improving AIs. You're failing to sell this argument to someone generally inclined to agree with you about the outcome.
BenjaminTodd 11y ago
The purpose of the profile isn't to argue a risk exists. We largely defer to the people we take to be experts on the issue, especially Nick Bostrom. We think he presents compelling arguments in Superintelligence, and although it's hard to say anything decisive in this area, if you think there's even modest uncertainty about whether AGI will be good or bad, it's worth doing more research into the risks. If you haven't read Bostrom's book yet, I'd really recommend it. http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/1501227742 http://www.amazon.com/Superintelligence-Dangers-Strategies-N...
- manish_gill 11y ago
  I read this book. Got about 4 chapters in before I had to give up at the sheer ridiculousness of the whole thing. The problem with this entire line of reasoning is that at this point it is nothing more an a thought experiment. Many of the key underlying assumptions that are required for Artificial General Intelligence simply have not been realised, and while Weak AI is progressing at a strong rate, we are hardly anywhere close to a place where we should start worrying about all this stuff. It's about as likely as a meteor hitting the planet and wiping out the entire human life. Possible? Sure. Should I be panicking about this right now? Nah. The book is nonsense. I'll start paying attention when someone who has real experience in the field of AI research (and I'm not talking charlatans like Yudkowsky here), but someone like say Norvig come out and say it's a reasonable concern today.
  - BenjaminTodd 11y ago
    The expert consensus says 10% chance of human-level AI in 10 years: http://www.givewell.org/labs/causes/ai-risk/ai-timelines http://www.givewell.org/labs/causes/ai-risk/ai-timelines Many computer science professors have publicly said they think AI poses significant risks. There's a list here: http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-risk/ http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-ri... Also see this open letter, signed by hundreds of experts: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/
    - 11y ago
ctogden 11y ago
What is HGI? Google/Bing/Wikipedia disambiguation are no help.
- andreyf 11y ago
  Human-level General Intelligence.
danieltillett 11y ago
Human level AI would have a profound impact on the value of human labor. If you don't consider everyone being unemployable an issue to society then I am not sure what would pose a significant risk. Most people that worry about this topic don't worry about human level AI, it is the vastly greater than human level AI that is the concern.
- punee 11y ago
  How does human level AI make everyone unemployable?
  - danieltillett 11y ago
    If the AI is at human level then it can do all human jobs. Given that humans require a minimum income to survive, if the total per hour cost of AI is below this level then all jobs will be taken by AI (why employ an expensive human when you can get a cheaper AI). This is what happened to human computers (people, mostly women, who performed mathematical calculations) in the 1950s when electronic computers became cheaper and faster per calculation. All those jobs ceased to exist. There are some caveats here. The cost of AI labor needs to be below that of the human minimum. Given there is no effective limit to how much AI labor you can create, the likely result is the cost of capital will fall to near zero making AI labor near free. No human will be able to compete against this except in providing personal services to the owners of capital in fluff jobs. The more interesting thing is that with the cost of capital and labour at near zero we should get massive deflation to the point that it will be very cheap to provide all people with a very high basic income that will make the need for employment unnecessary. This would be a great world to live in, but it would have a profound effect on the society unlike anything since the rise of agriculture.
    - punee 11y ago
      "If the AI is at human level then it can do all human jobs." I don't see how that follows. How does a computer program do manual labor?
throwaway808 11y ago
Two examples outside of science fiction: - The horse population declined dramatically after their work abilities were exceeded on every dimension by machines - Homo Neanderthalensis disappeared after the appearance of Homo Sapiens. I'm not saying these are perfect analogies. But you are saying they're completely irrelevant. Why is that?
- nairboon 11y ago
  Good old Homo Sapiens is already disappearing thanks to Homo Oeconomicus. I don't see a problem if we move on to Homo faber next.
threatofrain 11y ago
How are you so confident that the risks of machine learning are in the realm of science fiction? Machine learning, at least, has the plausible potential to permanently reduce demand on labor by capital owners. Undemanded laborers lose their ability to negotiate or strike as a power counterbalance to capital owners. At the very least, that future is too complicated to form such a confident opinion that the risks of machine learning are in the realm of science fiction.
- andreyf 11y ago
  There are certainly changes happening (and more on the way!) as a result of a combination of Moore's Law and ML research. I don't think they are inherently significantly more risky any more than other technical advances as long as we have a chance to think about and understand them. We definitely need to think through the social implications of computers. I think it's important to separate those real developments from the movie-plot-threat of a super-AI escaping human control.

rhaps0dy 11y ago

Be sure to de-lurk if that's the case! I guess. A lot of people already did. http://lukemuehlhauser.com/if-youre-an-ai-safety-lurker-now-would-be-a-good-time-to-de-lurk http://lukemuehlhauser.com/if-youre-an-ai-safety-lurker-now-...