7 ms·

Drawing a smiley face on a rock doesn't make it "happy", even if your primate neocortex tells you so. That's one of the best analogies we've got with regard to

by seiji 11y ago

Drawing a smiley face on a rock doesn't make it "happy", even if your primate neocortex tells you so. That's one of the best analogies we've got with regard to "deep learning" versus reality. People around here seem to think the AI apocalypse is 3-5 years away and are rushing to fund billion dollar "sentient rock" research.

Afforess 11y ago
An AI doesn't have to understand the world in order to destroy it. In fact, the less it understands, the more dangerous it could be.
- seiji 11y ago
  There's two lines of thought. Either a.) an AI will have the mental capacity of a 2 year old with the powers of a god — or — b.) an AI will have the mental capacity of a god (not a dumb Abrahamic god, but more like a universal atman) also combined with the powers of a god. If you're in the "2 year old with unlimited power" camp, nothing can save us and everything is futile and we should all just eat drink and be merry for tomorrow the AI kills us all. If you're in the AI-as-enlighened-buddha camp, the godlike AI will either save us all — or — just leave us alone to solve our own problems (while potentially locking out future godlike-AI development so we don't do too much runaway damage (eschaton, etc)).
  - Afforess 11y ago
    I wonder what exactly goes wrong when people think there are only two possibilities. Black and white aren't the only colors.
  - eli_gottlieb 11y ago
    The problem isn't that "AI" would have unlimited power. The problem is that foolish people want to put non-interpretable machine-learning models in charge of expensive and valuable stuff, because they see a model that sorta-if-you-squint-your-eyes seems clever-ish and hype themselves into calling it "AI".
  - fennecfoxen 11y ago
    Those are the only two lines of thought? Don't be INSANE. Where the hell do people get this shit from, bad sci-fi movies? I'm sorry, I don't really mean you're actually stupid or anything, it's just.... HOW ON EARTH could anyone with a passing familiarity with computer science possibly arrive at this sort of expectation? I don't understand!! The powers of a GOD? What does that even mean? No AI is going to be able to go "Let there be light!" and make there be light. Heck, no AI is going to be able to go "I will hack into this camera and spy on you!" without either spending the requisite CPU-hours to crack the passwords or encryption protecting it, or analyzing all its attack surface for weaknesses like a hacker. Computational complexity is REAL, P does not and never will equal NP (we just don't know how to prove it yet), and there are real physical limits on the computing power that you can fit inside a given volume and its energy budget. AT ITS BEST, an AI will have the same powers as a civilization of humans working together using computers the old-fashioned way, only faster.
    - mej10 11y ago
      Human brains are only so powerful. There are thoughts that literally cannot happen in a human brain due to size and speed limitations, and thus can never be thought by humans. AIs would have a much higher information density, approaching the theoretical limits for our universe. This alone is enough to reconsider the argument you are making. While an AI wouldn't have the power to fundamentally change the universe or defy computational complexity -- what they could do would be near enough to godlike in comparison to humans that such a fact barely matters.
    - AI_Overlord 11y ago
      An AI will be able to learn everything civilization knows. It will be able to work on problems, without rest, thousands of times faster than the entire human race. Quick example: In order to get the passwords this is what an AI can do. 1: build tiny robots. Something resembling a fruit fly. 2: robot fruit fly waits for appropriate person to use password. 3:see what password was used 4:mission accomplished This is just one way. I'm sure the AI can figure out simpler ways to get the passwords. Tell me this isn't god like. It Is like arguing that we'll never go to the moon. If evolution was able to invent intelligence so can we. And it will be a god. It is only a matter of time. And I'm going to build it. (Evil Maniacal Laugh)
  - deleted 11y ago
    [deleted]
- fennecfoxen 11y ago
  An AI doesn't have to understand the world in order to destroy it. It needs to be hooked up to the one technology which could credibly destroy the world: a nuclear weapons system. A very poorly designed nuclear weapons system that doesn't have any meaningful interlocks. Myself, I'm going to remain much more worried about the natural intelligences which have been hooked up to nuclear weapons systems.
  - crucialfelix 11y ago
    It doesn't need to be something big and scary like a nuclear weapons system. It could be hooked up to the stock market, and it could make entirely rational decisions based on its objective (profitable trades) and these actions can result in imbalances leading to famine in certain regions, increased pollution, unsustainable depletion of natural resources etc. We are already hooked up. The AIs are just amplifications of our own narrowly focused objectives.
    - dr_zoidberg 11y ago
      Then it has already happened[0], but few seem to recognize it. [0] https://en.wikipedia.org/wiki/2010_Flash_Crash https://en.wikipedia.org/wiki/2010_Flash_Crash
    - CuriouslyC 11y ago
      Algorithmic trading systems have already gone haywire, repeatedly in fact. All major stock markets "circuit breaker" systems in place to halt trading in the event of extreme volatility for specifically this reason. The sort of long range damaging activities you mention are unlikely though, as algorithmic trading systems in general take their long term cues from humans.

by seiji 11y ago

Afforess 11y ago

An AI doesn't have to understand the world in order to destroy it. In fact, the less it understands, the more dangerous it could be.

seiji 11y ago
There's two lines of thought. Either a.) an AI will have the mental capacity of a 2 year old with the powers of a god — or — b.) an AI will have the mental capacity of a god (not a dumb Abrahamic god, but more like a universal atman) also combined with the powers of a god. If you're in the "2 year old with unlimited power" camp, nothing can save us and everything is futile and we should all just eat drink and be merry for tomorrow the AI kills us all. If you're in the AI-as-enlighened-buddha camp, the godlike AI will either save us all — or — just leave us alone to solve our own problems (while potentially locking out future godlike-AI development so we don't do too much runaway damage (eschaton, etc)).
- Afforess 11y ago
  I wonder what exactly goes wrong when people think there are only two possibilities. Black and white aren't the only colors.
- eli_gottlieb 11y ago
  The problem isn't that "AI" would have unlimited power. The problem is that foolish people want to put non-interpretable machine-learning models in charge of expensive and valuable stuff, because they see a model that sorta-if-you-squint-your-eyes seems clever-ish and hype themselves into calling it "AI".
- fennecfoxen 11y ago
  Those are the only two lines of thought? Don't be INSANE. Where the hell do people get this shit from, bad sci-fi movies? I'm sorry, I don't really mean you're actually stupid or anything, it's just.... HOW ON EARTH could anyone with a passing familiarity with computer science possibly arrive at this sort of expectation? I don't understand!! The powers of a GOD? What does that even mean? No AI is going to be able to go "Let there be light!" and make there be light. Heck, no AI is going to be able to go "I will hack into this camera and spy on you!" without either spending the requisite CPU-hours to crack the passwords or encryption protecting it, or analyzing all its attack surface for weaknesses like a hacker. Computational complexity is REAL, P does not and never will equal NP (we just don't know how to prove it yet), and there are real physical limits on the computing power that you can fit inside a given volume and its energy budget. AT ITS BEST, an AI will have the same powers as a civilization of humans working together using computers the old-fashioned way, only faster.
  - mej10 11y ago
    Human brains are only so powerful. There are thoughts that literally cannot happen in a human brain due to size and speed limitations, and thus can never be thought by humans. AIs would have a much higher information density, approaching the theoretical limits for our universe. This alone is enough to reconsider the argument you are making. While an AI wouldn't have the power to fundamentally change the universe or defy computational complexity -- what they could do would be near enough to godlike in comparison to humans that such a fact barely matters.
  - AI_Overlord 11y ago
    An AI will be able to learn everything civilization knows. It will be able to work on problems, without rest, thousands of times faster than the entire human race. Quick example: In order to get the passwords this is what an AI can do. 1: build tiny robots. Something resembling a fruit fly. 2: robot fruit fly waits for appropriate person to use password. 3:see what password was used 4:mission accomplished This is just one way. I'm sure the AI can figure out simpler ways to get the passwords. Tell me this isn't god like. It Is like arguing that we'll never go to the moon. If evolution was able to invent intelligence so can we. And it will be a god. It is only a matter of time. And I'm going to build it. (Evil Maniacal Laugh)
- deleted 11y ago
  [deleted]
fennecfoxen 11y ago
An AI doesn't have to understand the world in order to destroy it. It needs to be hooked up to the one technology which could credibly destroy the world: a nuclear weapons system. A very poorly designed nuclear weapons system that doesn't have any meaningful interlocks. Myself, I'm going to remain much more worried about the natural intelligences which have been hooked up to nuclear weapons systems.
- crucialfelix 11y ago
  It doesn't need to be something big and scary like a nuclear weapons system. It could be hooked up to the stock market, and it could make entirely rational decisions based on its objective (profitable trades) and these actions can result in imbalances leading to famine in certain regions, increased pollution, unsustainable depletion of natural resources etc. We are already hooked up. The AIs are just amplifications of our own narrowly focused objectives.
  - dr_zoidberg 11y ago
    Then it has already happened[0], but few seem to recognize it. [0] https://en.wikipedia.org/wiki/2010_Flash_Crash https://en.wikipedia.org/wiki/2010_Flash_Crash
  - CuriouslyC 11y ago
    Algorithmic trading systems have already gone haywire, repeatedly in fact. All major stock markets "circuit breaker" systems in place to halt trading in the event of extreme volatility for specifically this reason. The sort of long range damaging activities you mention are unlikely though, as algorithmic trading systems in general take their long term cues from humans.