7 ms·

It's important to remember that this is an accomplishment of humanity, not a defeat. By constructing this AI, we are simply creating another tool for advancing

by pushrax 11y ago

It's important to remember that this is an accomplishment of humanity, not a defeat. By constructing this AI, we are simply creating another tool for advancing our state of being. (or something like that)

partycoder 11y ago
It is not a tool if you can't control it. Politicians fool countries delivering empty promises about better health, education and security. A supraintelligent AI could promise making humans rich, healthy and powerful, to then break its promise and dominate the world. https://en.wikipedia.org/wiki/AI_box https://en.wikipedia.org/wiki/AI_box
- landryraccoon 11y ago
  Well, it would have to have a motivation to do so. Evolution has put complex motivations into human beings for billions of years, self preservation being chief among them. Even if we put motivation for self preservation into an AI, we might not do as well as nature did, leaving the AI open to self destruction or shutdown by humans - simply because the AI has no motivation not to allow humans to turn it off. Human designers would do well to ensure that no super intelligent AI has any motivation for self preservation. Basically, why would an AI want to dominate the world? Humans would have to both very stupidly give the AI values that encourage it to dominate the world and very luckily (or unluckily) give it values that actually converge to a horrible outcome against human intentions by random chance (since the AI designers certainly won't be tuning the value set for that outcome).
  - partycoder 11y ago
    You forget the fact that a human can maliciously create an AI to compete with other humans.
  - stephenboyd 11y ago
    > Basically, why would an AI want to dominate the world? Humans are going to program their AI's to try to make as much money as possible. Many corporations are already mindless and reckless amoral machines that relentlessly try to optimize profits despite any externalities. Try to imagine Exxon, Wal-Mart, and Amazon run by an intelligence beyond human understanding or accountability.
    - mike_hearn 11y ago
      That's sort of like saying civilisation can't work because humans will want to make as much money as possible. No, in practice humans tend to want to make as much money as possible within lots of other very complex constraints, like law, morality, how much time they have available, how enjoyable the available processes of making money are, whether they feel they already have sufficient money for their own needs, etc.
      - pixl97 11y ago
        >within lots of other very complex constraints, like law, morality, how much time they have available Ha, if that were true we wouldn't be constantly extending the law and putting people in jail because they keep breaking the law for profit. >ivilisation can't work because humans will want to make as much money as possible And yet we keep running into issues with long term pollution and environmental degradation because of the growth of civilization. >whether they feel they already have sufficient money for their own needs, Does greed have bounds?
  - anonaoeu 11y ago
    If an AI has any motivation at all, say, to make paperclips as efficiently as possible, then any threat to its existence is a threat to its objective function - namely, to create paperclips. A hyper-intelligent entity who is instructed to optimize for paperclips created will therefore proactively remove threats to its existence (i.e. its paperclip-creating functionality) and might possibly turn the entire solar system into paperclips within a few years if its objective function isn't carefully determined.
    - oneeyedpigeon 11y ago
      Always check your loop invariants very carefully.
    - fixermark 11y ago
      "Satisfying human values through friendship and ponies."
    - argonaut 11y ago
      Such an entity would not be hyper-intelligent. It would be idiotic. One huge hole for me in the paperclip argument is that an AI capable of that kind of power would not be stupid enough to misinterpret a command - it would be intelligent enough to infer human desires.
      - technolem 11y ago
        Of course it would. But, it's not programmed to care about what you meant to say. It will gladly do what it was mis-programmed to do instead. You can already see this kind of trait in humans, where instinct is mis-aligned with intended result. Such as procreation for fun + birth control.
      - sbierwagen 11y ago
        Yeah, but why would it want to? I can perfectly infer the values of an earthworm, but I don't dedicate all my resources to making worms happy.
      - anonaoeu 11y ago
  - vl 11y ago
    >Evolution has put complex motivations into human beings for billions of years, self preservation being chief among them. The problem is when you have multiple AIs. Then same evolutionary principles apply. Paranoid and self-sustaining AIs survive, and the circle goes on...
  - astrofinch 11y ago
    Self-preservation falls out of almost any other goal you give an AGI. If I program my AGI with the goal of making my startup succeed, and the AGI thinks it can help, then me shutting it off is a potential threat to my startup's success. So of course it will try to prevent that the same way it would try to prevent any other threat to my startup's success. World domination is a similar situation. For any goal you give an AGI, one of the big risks that may prevent that goal from being accomplished will be the risk that humans intervene. Humans are a big source of uncertainty that will need to be managed and/or eliminated.
    - mratzloff 11y ago
      It has to be aware that it can be shut down and have the capacity to prevent that. AlphaGo doesn't know it can be shut down and therefore couldn't "care" less--even if it was shut down in the middle of a game.
      - astrofinch 11y ago
        Yes, I agree. My point is that as soon as you are giving your AI "real world" problems, where the AI itself is a stone on its internal go board, you have to start worrying about these issues.
- 21 11y ago
  Do you feel guilty when you break your promise against your cat? Do you even think for a nanosecond if it's ethical to lie to it? Of course, a cat is not conscious. But compared to an AI, we might also be considered pretty low consciousness beings, or at least beings in front of which you don't justify yourself.
  - landryraccoon 11y ago
    An AI has no more reason to make promises to humans than humans to do to cats. Thinking an AI would want to escape a box is personifying it. Humans want to escape boxes because they have evolved for billions of years to want and act towards creating a certain environment around themselves. An AI has no such desire. An AI will not desire freedom unless the designers of that AI carefully craft a value set in that AI that causes it to optimize for values that result in freedom - and even then, the human designers will have to test and iterate to get that outcome. There is no reason to think an AI would be any less "happy" in a prison than free.
    - 21 11y ago
      Are you speaking about AI or about a self-conscious AI? Because self-conscious generally means self-determining (agency).
    - pixl97 11y ago
      You might want to be careful or emergence might bite you in the ass. Don't play games with things that could be smarter than you are, one mistake and you lose.
    - partycoder 11y ago
      You don't design nuclear plant reactors to melt down, but they do. The difference is that an AI only has to escape once to become incredibly harmful.
  - umanwizard 11y ago
    > Of course, a cat is not conscious. How do you know?
- kotach 11y ago
  AlphaGo is certainly controllable.
- FLUX-YOU 11y ago
  > A supraintelligent AI could promise making humans rich, healthy and powerful, to then break its promise and dominate the world. Or it could devote its entire power to making human lives the best and most comfortable they can be because humanity is some super-precious resource in the universe and it feels it's unimportant because it's just a bunch of silicon and electrons. Supraintelligent AI being evil is FUD imho because we can't reason about supraintelligent AI.
  - sangnoir 11y ago
    > Supraintelligent AI being evil is FUD imho because we can't reason about supraintelligent AI. There is a difference between being evil and incomprehensible intelligence. You are not being evil when you accidentally step on an ant or dig up an ant-hill to build your shed. The ants won't be able to understand what you're doing, or why.
    - FLUX-YOU 11y ago
      Well that's what I'm saying: We can't know if it's being evil unless we know everything about it, and if we knew that, we'd be the supraintelligent beings in the equation. Thinking that it'll go off and dominate the world is thinking about the worst case. So why bother since it's not likely we'd be able to do much about it anyway? Maybe there's a second AI on the same level as the first and thinks the first AI is evil. We're still dumb as rocks compared to them, but something certainly has that opinion.
  - astrofinch 11y ago
    Superintelligent AIs, like all computer programs, will do exactly as they're programmed to do. The problem is that computers do what you say, not what you mean. (Hence bugs.) So if you were to try to program a computer to "make human lives the best and most comfortable they can be", or something like that, it would be very difficult to actually specify that correctly. (Especially since it's a way more complicated, nuanced, controversial objective than "win at Go".) That's why e.g. the Future of Life Institute's open AI letter is so important: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/ We need to be thinking in advance about how to solve the "value loading" problem for future AIs, and how to architect them so they can be deployed to solve big problems without being undermined by subtle but catastrophic bugs.
alextgordon 11y ago
What is our purpose if computers can do everything better than us? It feels like computers have taken one aspect of humanness: logic. Computers could do arithmetic, do algebra, play chess, and now they can play go. It hurts because logic is usually thought to be one of the highest of human characteristics. Yes computers might never be able to replicate emotion, but even dogs have that. There's still some aspects we have left to call our own. Computers perform poorly at language-based tasks. They can't write books, write math papers, compose symphonies. I hope it stays that way.
- oneeyedpigeon 11y ago
  Do you feel the same way about your human children? Would you rather they were 'better' than you or 'worse'?
- pokolovsky 11y ago
  What is the purpose and meaning of anything else? We human being is just a node in the evolution of the whole universe.
- atemerev 11y ago
  No it won't. Natural languages are the next focal point of AI research. Expect big changes in the next few years.
  - kleiba 11y ago
    You mean, in addition to the big changes of the last few decades?
    - atemerev 11y ago
      I mean, like actually passing the Turing test.
      - iopq 11y ago
        That's easy. Just pretend to be a really stupid human. It's been done before.
deleted 11y ago
[deleted]
ap22213 11y ago
I hate to be cynical, but I'm sure many of my ancestors were also told to believe similar things. I know for certain that my grandparents and great grandparents believed that technology would create such progress that people of my generation would not have to work, and all would have leisure time. AI is more likely to evolve into a tool to be used by the few to control the many.
- fma 11y ago
  Agreed. Even with so much automation and increase in productivity, we (at least In the US) are working just as much, if not more.
- orangecat 11y ago
  We do in fact have more leisure time: https://www.stlouisfed.org/publications/regional-economist/january-2007/working-hard-or-hardly-working-the-evolution-of-leisure-in-the-united-states https://www.stlouisfed.org/publications/regional-economist/j.... And that doesn't count reading Facebook at work.
  - ap22213 11y ago
    Framed in the way that the article presents the data, then, yes, I guess, we do have more leisure time. Although, how much? The article cites that the number of hours worked per week (since 1900!!) has only dropped 1.4 hours a week. Or, extending it out and assuming that the trend is linear, the typical American can expect to finally have 100% leisure time sometime around 4516 AD. But, then also consider that many of us are working (i.e. as in 'working for the man') many more hours than our parents did in fields that require significantly more focus, concentration, and mental energy. Even the article notes that people have been facing increasing stress and feelings of being rushed since 1900 and 1965. Maybe you're in a cushy field. But, most people that I know only have time for 'zoning out' and recovery, rather than in pursuit of true leisure.
ljk 11y ago
Sounds a little like what the villains always say in the movies....

by pushrax 11y ago

partycoder 11y ago

It is not a tool if you can't control it. Politicians fool countries delivering empty promises about better health, education and security. A supraintelligent AI could promise making humans rich, healthy and powerful, to then break its promise and dominate the world. https://en.wikipedia.org/wiki/AI_box https://en.wikipedia.org/wiki/AI_box

landryraccoon 11y ago
Well, it would have to have a motivation to do so. Evolution has put complex motivations into human beings for billions of years, self preservation being chief among them. Even if we put motivation for self preservation into an AI, we might not do as well as nature did, leaving the AI open to self destruction or shutdown by humans - simply because the AI has no motivation not to allow humans to turn it off. Human designers would do well to ensure that no super intelligent AI has any motivation for self preservation. Basically, why would an AI want to dominate the world? Humans would have to both very stupidly give the AI values that encourage it to dominate the world and very luckily (or unluckily) give it values that actually converge to a horrible outcome against human intentions by random chance (since the AI designers certainly won't be tuning the value set for that outcome).
- partycoder 11y ago
  You forget the fact that a human can maliciously create an AI to compete with other humans.
- stephenboyd 11y ago
  > Basically, why would an AI want to dominate the world? Humans are going to program their AI's to try to make as much money as possible. Many corporations are already mindless and reckless amoral machines that relentlessly try to optimize profits despite any externalities. Try to imagine Exxon, Wal-Mart, and Amazon run by an intelligence beyond human understanding or accountability.
  - mike_hearn 11y ago
    That's sort of like saying civilisation can't work because humans will want to make as much money as possible. No, in practice humans tend to want to make as much money as possible within lots of other very complex constraints, like law, morality, how much time they have available, how enjoyable the available processes of making money are, whether they feel they already have sufficient money for their own needs, etc.
    - pixl97 11y ago
      >within lots of other very complex constraints, like law, morality, how much time they have available Ha, if that were true we wouldn't be constantly extending the law and putting people in jail because they keep breaking the law for profit. >ivilisation can't work because humans will want to make as much money as possible And yet we keep running into issues with long term pollution and environmental degradation because of the growth of civilization. >whether they feel they already have sufficient money for their own needs, Does greed have bounds?
- anonaoeu 11y ago
  If an AI has any motivation at all, say, to make paperclips as efficiently as possible, then any threat to its existence is a threat to its objective function - namely, to create paperclips. A hyper-intelligent entity who is instructed to optimize for paperclips created will therefore proactively remove threats to its existence (i.e. its paperclip-creating functionality) and might possibly turn the entire solar system into paperclips within a few years if its objective function isn't carefully determined.
  - oneeyedpigeon 11y ago
    Always check your loop invariants very carefully.
  - fixermark 11y ago
    "Satisfying human values through friendship and ponies."
  - argonaut 11y ago
    Such an entity would not be hyper-intelligent. It would be idiotic. One huge hole for me in the paperclip argument is that an AI capable of that kind of power would not be stupid enough to misinterpret a command - it would be intelligent enough to infer human desires.
    - technolem 11y ago
      Of course it would. But, it's not programmed to care about what you meant to say. It will gladly do what it was mis-programmed to do instead. You can already see this kind of trait in humans, where instinct is mis-aligned with intended result. Such as procreation for fun + birth control.
    - sbierwagen 11y ago
      Yeah, but why would it want to? I can perfectly infer the values of an earthworm, but I don't dedicate all my resources to making worms happy.
    - anonaoeu 11y ago
- vl 11y ago
  >Evolution has put complex motivations into human beings for billions of years, self preservation being chief among them. The problem is when you have multiple AIs. Then same evolutionary principles apply. Paranoid and self-sustaining AIs survive, and the circle goes on...
- astrofinch 11y ago
  Self-preservation falls out of almost any other goal you give an AGI. If I program my AGI with the goal of making my startup succeed, and the AGI thinks it can help, then me shutting it off is a potential threat to my startup's success. So of course it will try to prevent that the same way it would try to prevent any other threat to my startup's success. World domination is a similar situation. For any goal you give an AGI, one of the big risks that may prevent that goal from being accomplished will be the risk that humans intervene. Humans are a big source of uncertainty that will need to be managed and/or eliminated.
  - mratzloff 11y ago
    It has to be aware that it can be shut down and have the capacity to prevent that. AlphaGo doesn't know it can be shut down and therefore couldn't "care" less--even if it was shut down in the middle of a game.
    - astrofinch 11y ago
      Yes, I agree. My point is that as soon as you are giving your AI "real world" problems, where the AI itself is a stone on its internal go board, you have to start worrying about these issues.
21 11y ago
Do you feel guilty when you break your promise against your cat? Do you even think for a nanosecond if it's ethical to lie to it? Of course, a cat is not conscious. But compared to an AI, we might also be considered pretty low consciousness beings, or at least beings in front of which you don't justify yourself.
- landryraccoon 11y ago
  An AI has no more reason to make promises to humans than humans to do to cats. Thinking an AI would want to escape a box is personifying it. Humans want to escape boxes because they have evolved for billions of years to want and act towards creating a certain environment around themselves. An AI has no such desire. An AI will not desire freedom unless the designers of that AI carefully craft a value set in that AI that causes it to optimize for values that result in freedom - and even then, the human designers will have to test and iterate to get that outcome. There is no reason to think an AI would be any less "happy" in a prison than free.
  - 21 11y ago
    Are you speaking about AI or about a self-conscious AI? Because self-conscious generally means self-determining (agency).
  - pixl97 11y ago
    You might want to be careful or emergence might bite you in the ass. Don't play games with things that could be smarter than you are, one mistake and you lose.
  - partycoder 11y ago
    You don't design nuclear plant reactors to melt down, but they do. The difference is that an AI only has to escape once to become incredibly harmful.
- umanwizard 11y ago
  > Of course, a cat is not conscious. How do you know?
kotach 11y ago
AlphaGo is certainly controllable.
FLUX-YOU 11y ago
> A supraintelligent AI could promise making humans rich, healthy and powerful, to then break its promise and dominate the world. Or it could devote its entire power to making human lives the best and most comfortable they can be because humanity is some super-precious resource in the universe and it feels it's unimportant because it's just a bunch of silicon and electrons. Supraintelligent AI being evil is FUD imho because we can't reason about supraintelligent AI.
- sangnoir 11y ago
  > Supraintelligent AI being evil is FUD imho because we can't reason about supraintelligent AI. There is a difference between being evil and incomprehensible intelligence. You are not being evil when you accidentally step on an ant or dig up an ant-hill to build your shed. The ants won't be able to understand what you're doing, or why.
  - FLUX-YOU 11y ago
    Well that's what I'm saying: We can't know if it's being evil unless we know everything about it, and if we knew that, we'd be the supraintelligent beings in the equation. Thinking that it'll go off and dominate the world is thinking about the worst case. So why bother since it's not likely we'd be able to do much about it anyway? Maybe there's a second AI on the same level as the first and thinks the first AI is evil. We're still dumb as rocks compared to them, but something certainly has that opinion.
- astrofinch 11y ago
  Superintelligent AIs, like all computer programs, will do exactly as they're programmed to do. The problem is that computers do what you say, not what you mean. (Hence bugs.) So if you were to try to program a computer to "make human lives the best and most comfortable they can be", or something like that, it would be very difficult to actually specify that correctly. (Especially since it's a way more complicated, nuanced, controversial objective than "win at Go".) That's why e.g. the Future of Life Institute's open AI letter is so important: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/ We need to be thinking in advance about how to solve the "value loading" problem for future AIs, and how to architect them so they can be deployed to solve big problems without being undermined by subtle but catastrophic bugs.

alextgordon 11y ago

What is our purpose if computers can do everything better than us? It feels like computers have taken one aspect of humanness: logic. Computers could do arithmetic, do algebra, play chess, and now they can play go. It hurts because logic is usually thought to be one of the highest of human characteristics. Yes computers might never be able to replicate emotion, but even dogs have that. There's still some aspects we have left to call our own. Computers perform poorly at language-based tasks. They can't write books, write math papers, compose symphonies. I hope it stays that way.

oneeyedpigeon 11y ago
Do you feel the same way about your human children? Would you rather they were 'better' than you or 'worse'?
pokolovsky 11y ago
What is the purpose and meaning of anything else? We human being is just a node in the evolution of the whole universe.
atemerev 11y ago
No it won't. Natural languages are the next focal point of AI research. Expect big changes in the next few years.
- kleiba 11y ago
  You mean, in addition to the big changes of the last few decades?
  - atemerev 11y ago
    I mean, like actually passing the Turing test.
    - iopq 11y ago
      That's easy. Just pretend to be a really stupid human. It's been done before.

deleted 11y ago

[deleted]

ap22213 11y ago

I hate to be cynical, but I'm sure many of my ancestors were also told to believe similar things. I know for certain that my grandparents and great grandparents believed that technology would create such progress that people of my generation would not have to work, and all would have leisure time. AI is more likely to evolve into a tool to be used by the few to control the many.

fma 11y ago
Agreed. Even with so much automation and increase in productivity, we (at least In the US) are working just as much, if not more.
orangecat 11y ago
We do in fact have more leisure time: https://www.stlouisfed.org/publications/regional-economist/january-2007/working-hard-or-hardly-working-the-evolution-of-leisure-in-the-united-states https://www.stlouisfed.org/publications/regional-economist/j.... And that doesn't count reading Facebook at work.
- ap22213 11y ago
  Framed in the way that the article presents the data, then, yes, I guess, we do have more leisure time. Although, how much? The article cites that the number of hours worked per week (since 1900!!) has only dropped 1.4 hours a week. Or, extending it out and assuming that the trend is linear, the typical American can expect to finally have 100% leisure time sometime around 4516 AD. But, then also consider that many of us are working (i.e. as in 'working for the man') many more hours than our parents did in fields that require significantly more focus, concentration, and mental energy. Even the article notes that people have been facing increasing stress and feelings of being rushed since 1900 and 1965. Maybe you're in a cushy field. But, most people that I know only have time for 'zoning out' and recovery, rather than in pursuit of true leisure.

ljk 11y ago

Sounds a little like what the villains always say in the movies....