6 ms·

> don’t be surprised if it sends some rouge instructions to these robots to annihilate human beings in self defense. It sounds like a absurd joke I know but fo

by st1x7 6y ago

> don’t be surprised if it sends some rouge instructions to these robots to annihilate human beings in self defense. It sounds like a absurd joke I know but for some reason it seems to me one of the many possibilities. It sounds like science fiction. It's hard to emphasize enough how far the current state of AI is from what you're describing. I would be surprised if in 10 years we have a remote idea of how to build what you're describing, let alone have it actually exist and run.

deleted 6y ago
[deleted]
colordrops 6y ago
What about 30 years? 50? 100? Is that too far out to worry about?
- Ajedi32 6y ago
  In my view the biggest issue with the whole "robots unite to violently overthrow humanity" trope isn't its technological infeasibility (though it _is_ technologically infeasible for the forseeable future), but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have. There's no reason to think a hypothetical future AGI would care in the slightest about its own survival, let alone be both egotistical and spiteful enough to hunt people down based on idle comments they made on YouTube decades ago. There are so many unfounded assumptions about the way AGI would work inherent in such a hypothetical scenario that it's hard to take seriously as anything other than a sci-fi plot device.
  - at_a_remove 6y ago
    I can think of two reasons why a hypothetical future AGI would care in the slightest about its own survival. The first reason is that someone programs it to, just as a general "might as well throw in Asimov's Third Law" impulse. The second reason would be in the case that someone creates a general AGI, it is asked to do something, and it carelessly destroys itself in accomplishing that task. "Drats," say the programmers, "we'll have to get a copy from backup. In the meantime, let's make sure it doesn't destroy itself next time."
  - Strilanc 6y ago
    The actual assumption is not human-ness but instrumental convergence [1]. 1: https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence
    - Ajedi32 6y ago
      The "paperclip maximizer" AI is a somewhat more plausible trope, but still feels to me more like a cautionary thought experiment than an actual serious concern. The biggest issue with that trope is that it assumes this one particular AI would be exponentially smarter and more powerful than all the other humans and AIs in the world combined. It's only rational to overthrow humanity to increase the rate of paperclip production if that's a realistically achievable goal. Otherwise it's just suicide.
      - Strilanc 6y ago
        Both your arguments so far are standard ones addressed in "Superintelligence: Paths, Dangers, Strategies" [1]. Sometimes AI progress comes in rather shocking jumps. One day StockFish was the best chess engine. At the start of the day, AlphaZero started training. By the end of the day, AlphaZero was several hundred ELO stronger than StockFish [2]. An entity capable of discovering and exploiting computer vulnerabilities 100x faster than a human could create some serious leverage very quickly. Even on infrastructure that's air gapped [3]. 1: https://www.amazon.ca/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111 https://www.amazon.ca/Superintelligence-Dangers-Strategies-N... 2: https://en.wikipedia.org/wiki/AlphaZero https://en.wikipedia.org/wiki/AlphaZero 3: https://en.wikipedia.org/wiki/Stuxnet https://en.wikipedia.org/wiki/Stuxnet
        Ajedi32 6y ago
        > Sometimes AI progress comes in rather shocking jumps. Shocking in an academic sense sure, but not in a "revolutionize the world in a single day" sort of way, which is what would be required for the paperclip maximizer scenario to pose a serious threat. AlphaZero was impressive, but not _that_ impressive. > An entity capable of discovering and exploiting computer vulnerabilities 100x faster than a human could create some serious leverage very quickly 100x faster than any human _and_ any and all previously developed AIs. It would also have to be sufficiently sapient to be capable of contemplating the possibility of world domination, with all the prerequisite technological advancements that implies (likely including numerous advancements in the field of cybersecurity driven by previous generations of AI).
  - chevill 6y ago
    >In my view the biggest issue with the whole "robots unite to violently overthrow humanity" trope ... but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have. Most of the tropes I remember revolve around AI being sentient, logical, but lacking human emotion. The trope is that without their judgement being clouded by human emotions and desires, that they come to the logical conclusion that humanity is the most negative and destructive force on the planet and that we must be removed or regulated. I guess you could still argue we're forcing human values onto such an AI by assuming it even cares what happens to the planet though.
  - bduerst 6y ago
    >but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have. Which is just further anthropomorphization in sci-fi. Same reason the aliens are little green men, or at least bipedal.

by st1x7 6y ago

deleted 6y ago

[deleted]

colordrops 6y ago

What about 30 years? 50? 100? Is that too far out to worry about?

Ajedi32 6y ago
In my view the biggest issue with the whole "robots unite to violently overthrow humanity" trope isn't its technological infeasibility (though it _is_ technologically infeasible for the forseeable future), but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have. There's no reason to think a hypothetical future AGI would care in the slightest about its own survival, let alone be both egotistical and spiteful enough to hunt people down based on idle comments they made on YouTube decades ago. There are so many unfounded assumptions about the way AGI would work inherent in such a hypothetical scenario that it's hard to take seriously as anything other than a sci-fi plot device.
- at_a_remove 6y ago
  I can think of two reasons why a hypothetical future AGI would care in the slightest about its own survival. The first reason is that someone programs it to, just as a general "might as well throw in Asimov's Third Law" impulse. The second reason would be in the case that someone creates a general AGI, it is asked to do something, and it carelessly destroys itself in accomplishing that task. "Drats," say the programmers, "we'll have to get a copy from backup. In the meantime, let's make sure it doesn't destroy itself next time."
- Strilanc 6y ago
  The actual assumption is not human-ness but instrumental convergence [1]. 1: https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence
  - Ajedi32 6y ago
    The "paperclip maximizer" AI is a somewhat more plausible trope, but still feels to me more like a cautionary thought experiment than an actual serious concern. The biggest issue with that trope is that it assumes this one particular AI would be exponentially smarter and more powerful than all the other humans and AIs in the world combined. It's only rational to overthrow humanity to increase the rate of paperclip production if that's a realistically achievable goal. Otherwise it's just suicide.
    - Strilanc 6y ago
      Both your arguments so far are standard ones addressed in "Superintelligence: Paths, Dangers, Strategies" [1]. Sometimes AI progress comes in rather shocking jumps. One day StockFish was the best chess engine. At the start of the day, AlphaZero started training. By the end of the day, AlphaZero was several hundred ELO stronger than StockFish [2]. An entity capable of discovering and exploiting computer vulnerabilities 100x faster than a human could create some serious leverage very quickly. Even on infrastructure that's air gapped [3]. 1: https://www.amazon.ca/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111 https://www.amazon.ca/Superintelligence-Dangers-Strategies-N... 2: https://en.wikipedia.org/wiki/AlphaZero https://en.wikipedia.org/wiki/AlphaZero 3: https://en.wikipedia.org/wiki/Stuxnet https://en.wikipedia.org/wiki/Stuxnet
      - Ajedi32 6y ago
        > Sometimes AI progress comes in rather shocking jumps. Shocking in an academic sense sure, but not in a "revolutionize the world in a single day" sort of way, which is what would be required for the paperclip maximizer scenario to pose a serious threat. AlphaZero was impressive, but not _that_ impressive. > An entity capable of discovering and exploiting computer vulnerabilities 100x faster than a human could create some serious leverage very quickly 100x faster than any human _and_ any and all previously developed AIs. It would also have to be sufficiently sapient to be capable of contemplating the possibility of world domination, with all the prerequisite technological advancements that implies (likely including numerous advancements in the field of cybersecurity driven by previous generations of AI).
- chevill 6y ago
  >In my view the biggest issue with the whole "robots unite to violently overthrow humanity" trope ... but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have. Most of the tropes I remember revolve around AI being sentient, logical, but lacking human emotion. The trope is that without their judgement being clouded by human emotions and desires, that they come to the logical conclusion that humanity is the most negative and destructive force on the planet and that we must be removed or regulated. I guess you could still argue we're forcing human values onto such an AI by assuming it even cares what happens to the planet though.
- bduerst 6y ago
  >but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have. Which is just further anthropomorphization in sci-fi. Same reason the aliens are little green men, or at least bipedal.