4 ms·

I disagree with the premise. AI isn't like a nuclear warhead. It's not a machine for pure destruction. AI can be used as much to generate welfare as to damage -

I disagree with the premise. AI isn't like a nuclear warhead. It's not a machine for pure destruction. AI can be used as much to generate welfare as to damage - it's all in the application. Sharing methods benefits at least as much as it could hurt.

FeepingCreature 9y ago
I think it can be used as much to generate welfare as destruction when it goes right, but the cases where it goes wrong are both plentiful and heavily biased towards destruction.
- dsacco 9y ago
  What are a few examples of those cases, exactly?
  - s2g 9y ago
    Terminator, 2001, I Robot, ... oh wait you probably meant in reality. sorry. edit: Oh and how could I forget the Matrix. Which of course we all totally live in.
    - Robotbeat 9y ago
      Before nuclear weapons were built, the only examples of their large-scale destruction were in fiction. The World Set Free by H.G. Wells is the main example I can think of: https://en.wikipedia.org/wiki/The_World_Set_Free https://en.wikipedia.org/wiki/The_World_Set_Free It got some of the details wrong (i.e. the atomic bombs exploded over a long period of time vs instantly), but the consequences were fairly accurate. If you're going to wait until all the actual details and examples happen in reality, then it's too late to have much insight, especially if we're talking about something capable of self-replication like a virus or artificial intelligence. The nature of the problem REQUIRES anticipation instead of mere reaction.
      - dsacco 9y ago
        This is survivorship bias. How many weapons of mass destruction have been foretold by science fiction that have never been invented? More generally, how much technology has been written about in science fiction that we have only the faintest hope of ever achieving?
        Robotbeat 9y ago
        Maybe one in five, depending on how you define it? Look, it comes down to this: Is there something innate about our intelligence that makes it impossible to match except through human brains? That strikes me as something akin to spiritualism. The answer is almost certainly "no." We're not talking about warping space through some hypothetical state of matter with laws of physics different from our own. We're talking about at least human-level intelligence, which is something we already have an existence proof of (us). And humans are the most dangerous force on the planet. No animal (beside maybe microscopic organisms) stand a chance against a group determined humans. We're nearly unstoppable due to our intelligence. Our we so unique? Is it so impossible that machines could some day (perhaps in our lifetimes) be built which achieve human-level intelligence? Considering the computer advances that have already been achieved, it is clearly a real possibility if not a certainty. Human-level intelligence is perhaps the most powerful (thus dangerous) thing on this planet. Making a machine that is at least as intelligent (and perhaps much more so!) is clearly something that could be incredibly dangerous, and we know such a thing is physically possible, unlike many things in science fiction.
  - eli_gottlieb 9y ago
    See: basically every instance of Goodhart's Law. Remember, "AI" in these terms is basically just stochastic optimization (ie: machine learning), so the whole range of problems with throwing mathematical optimization at a vaguely-defined intuitive problem crop up. And there are a lot of those.
    - dsacco 9y ago
      > See: basically every instance of Goodhart's Law No, I asked for specific examples. This is part of the handwaving I'm talking about. Can you give me something other than the maniacal paperclip optimizer?
      - rspeer 9y ago
        YouTube video recommendations, which converge on recommending conspiracy theories and shocking videos to easily manipulated people, especially children. I am not worried about literal paperclip maximizers, but this may be the closest real thing to that parable. The hypothesis isn't that YouTube's recommender system didn't work -- it's that it worked too well at its assigned task of maximizing view time, and we humans are finally realizing that maximizing view time was not what we actually wanted.
        p1esk 9y ago
        Do you think limiting research on recommender systems (or limiting access to such research) would help in this case? What would be a solution to YouTube recommendation problem?
        colah3 9y ago
        It seems like OpenAI's Human Feedback research (in collaboration with DeepMind) is targeted at this sort of thing. They try to use human feedback to create more nuanced and aligned objectives. https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/ https://blog.openai.com/deep-reinforcement-learning-from-hum...
eb3c90 9y ago
I think I disagree. When you get past AGI to where it can do things that humans can't, a lot of current safeguards haven't been designed with it in mind. So things might be vulnerable. The kinds of things I'm thinking about are the various countries nuclear arsenals might not be safe from actors with very advanced AI. This I think is the potential source of existential risk, in my opinion. So it could hurt a hell of a lot. So I'm of the responsible disclosure point of view. You ask "Would releasing AI advancement X mess up someones security/economy". If so, you help them patch it before releasing it to the general public. The majority of advancements aren't like that and they won't be for a while.
- heurist 9y ago
  The world will evolve with the development of AI; I'm not so concerned with limitations of current safeguards.
  - sanxiyn 9y ago
    The world did evolve with the development of nuclear weapon, but between 1945 and 1949 US was the sole nuclear power and could preemptively attack USSR as John von Neumann proposed. That's 4 years! I suspect such window will recur with AI.

FeepingCreature 9y ago

I think it can be used as much to generate welfare as destruction when it goes right, but the cases where it goes wrong are both plentiful and heavily biased towards destruction.

dsacco 9y ago
What are a few examples of those cases, exactly?
- s2g 9y ago
  Terminator, 2001, I Robot, ... oh wait you probably meant in reality. sorry. edit: Oh and how could I forget the Matrix. Which of course we all totally live in.
  - Robotbeat 9y ago
    Before nuclear weapons were built, the only examples of their large-scale destruction were in fiction. The World Set Free by H.G. Wells is the main example I can think of: https://en.wikipedia.org/wiki/The_World_Set_Free https://en.wikipedia.org/wiki/The_World_Set_Free It got some of the details wrong (i.e. the atomic bombs exploded over a long period of time vs instantly), but the consequences were fairly accurate. If you're going to wait until all the actual details and examples happen in reality, then it's too late to have much insight, especially if we're talking about something capable of self-replication like a virus or artificial intelligence. The nature of the problem REQUIRES anticipation instead of mere reaction.
    - dsacco 9y ago
      This is survivorship bias. How many weapons of mass destruction have been foretold by science fiction that have never been invented? More generally, how much technology has been written about in science fiction that we have only the faintest hope of ever achieving?
      - Robotbeat 9y ago
        Maybe one in five, depending on how you define it? Look, it comes down to this: Is there something innate about our intelligence that makes it impossible to match except through human brains? That strikes me as something akin to spiritualism. The answer is almost certainly "no." We're not talking about warping space through some hypothetical state of matter with laws of physics different from our own. We're talking about at least human-level intelligence, which is something we already have an existence proof of (us). And humans are the most dangerous force on the planet. No animal (beside maybe microscopic organisms) stand a chance against a group determined humans. We're nearly unstoppable due to our intelligence. Our we so unique? Is it so impossible that machines could some day (perhaps in our lifetimes) be built which achieve human-level intelligence? Considering the computer advances that have already been achieved, it is clearly a real possibility if not a certainty. Human-level intelligence is perhaps the most powerful (thus dangerous) thing on this planet. Making a machine that is at least as intelligent (and perhaps much more so!) is clearly something that could be incredibly dangerous, and we know such a thing is physically possible, unlike many things in science fiction.
- eli_gottlieb 9y ago
  See: basically every instance of Goodhart's Law. Remember, "AI" in these terms is basically just stochastic optimization (ie: machine learning), so the whole range of problems with throwing mathematical optimization at a vaguely-defined intuitive problem crop up. And there are a lot of those.
  - dsacco 9y ago
    > See: basically every instance of Goodhart's Law No, I asked for specific examples. This is part of the handwaving I'm talking about. Can you give me something other than the maniacal paperclip optimizer?
    - rspeer 9y ago
      YouTube video recommendations, which converge on recommending conspiracy theories and shocking videos to easily manipulated people, especially children. I am not worried about literal paperclip maximizers, but this may be the closest real thing to that parable. The hypothesis isn't that YouTube's recommender system didn't work -- it's that it worked too well at its assigned task of maximizing view time, and we humans are finally realizing that maximizing view time was not what we actually wanted.
      - p1esk 9y ago
        Do you think limiting research on recommender systems (or limiting access to such research) would help in this case? What would be a solution to YouTube recommendation problem?
        colah3 9y ago
        It seems like OpenAI's Human Feedback research (in collaboration with DeepMind) is targeted at this sort of thing. They try to use human feedback to create more nuanced and aligned objectives. https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/ https://blog.openai.com/deep-reinforcement-learning-from-hum...

eb3c90 9y ago

I think I disagree. When you get past AGI to where it can do things that humans can't, a lot of current safeguards haven't been designed with it in mind. So things might be vulnerable. The kinds of things I'm thinking about are the various countries nuclear arsenals might not be safe from actors with very advanced AI. This I think is the potential source of existential risk, in my opinion. So it could hurt a hell of a lot. So I'm of the responsible disclosure point of view. You ask "Would releasing AI advancement X mess up someones security/economy". If so, you help them patch it before releasing it to the general public. The majority of advancements aren't like that and they won't be for a while.

heurist 9y ago
The world will evolve with the development of AI; I'm not so concerned with limitations of current safeguards.
- sanxiyn 9y ago
  The world did evolve with the development of nuclear weapon, but between 1945 and 1949 US was the sole nuclear power and could preemptively attack USSR as John von Neumann proposed. That's 4 years! I suspect such window will recur with AI.