3 ms·

What are a few examples of those cases, exactly?

by dsacco 9y ago

s2g 9y ago
Terminator, 2001, I Robot, ... oh wait you probably meant in reality. sorry. edit: Oh and how could I forget the Matrix. Which of course we all totally live in.
- Robotbeat 9y ago
  Before nuclear weapons were built, the only examples of their large-scale destruction were in fiction. The World Set Free by H.G. Wells is the main example I can think of: https://en.wikipedia.org/wiki/The_World_Set_Free https://en.wikipedia.org/wiki/The_World_Set_Free It got some of the details wrong (i.e. the atomic bombs exploded over a long period of time vs instantly), but the consequences were fairly accurate. If you're going to wait until all the actual details and examples happen in reality, then it's too late to have much insight, especially if we're talking about something capable of self-replication like a virus or artificial intelligence. The nature of the problem REQUIRES anticipation instead of mere reaction.
  - dsacco 9y ago
    This is survivorship bias. How many weapons of mass destruction have been foretold by science fiction that have never been invented? More generally, how much technology has been written about in science fiction that we have only the faintest hope of ever achieving?
    - Robotbeat 9y ago
      Maybe one in five, depending on how you define it? Look, it comes down to this: Is there something innate about our intelligence that makes it impossible to match except through human brains? That strikes me as something akin to spiritualism. The answer is almost certainly "no." We're not talking about warping space through some hypothetical state of matter with laws of physics different from our own. We're talking about at least human-level intelligence, which is something we already have an existence proof of (us). And humans are the most dangerous force on the planet. No animal (beside maybe microscopic organisms) stand a chance against a group determined humans. We're nearly unstoppable due to our intelligence. Our we so unique? Is it so impossible that machines could some day (perhaps in our lifetimes) be built which achieve human-level intelligence? Considering the computer advances that have already been achieved, it is clearly a real possibility if not a certainty. Human-level intelligence is perhaps the most powerful (thus dangerous) thing on this planet. Making a machine that is at least as intelligent (and perhaps much more so!) is clearly something that could be incredibly dangerous, and we know such a thing is physically possible, unlike many things in science fiction.
eli_gottlieb 9y ago
See: basically every instance of Goodhart's Law. Remember, "AI" in these terms is basically just stochastic optimization (ie: machine learning), so the whole range of problems with throwing mathematical optimization at a vaguely-defined intuitive problem crop up. And there are a lot of those.
- dsacco 9y ago
  > See: basically every instance of Goodhart's Law No, I asked for specific examples. This is part of the handwaving I'm talking about. Can you give me something other than the maniacal paperclip optimizer?
  - rspeer 9y ago
    YouTube video recommendations, which converge on recommending conspiracy theories and shocking videos to easily manipulated people, especially children. I am not worried about literal paperclip maximizers, but this may be the closest real thing to that parable. The hypothesis isn't that YouTube's recommender system didn't work -- it's that it worked too well at its assigned task of maximizing view time, and we humans are finally realizing that maximizing view time was not what we actually wanted.
    - p1esk 9y ago
      Do you think limiting research on recommender systems (or limiting access to such research) would help in this case? What would be a solution to YouTube recommendation problem?
      - colah3 9y ago
        It seems like OpenAI's Human Feedback research (in collaboration with DeepMind) is targeted at this sort of thing. They try to use human feedback to create more nuanced and aligned objectives. https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/ https://blog.openai.com/deep-reinforcement-learning-from-hum...

What are a few examples of those cases, exactly?

by dsacco 9y ago

What are a few examples of those cases, exactly?

s2g 9y ago

Terminator, 2001, I Robot, ... oh wait you probably meant in reality. sorry. edit: Oh and how could I forget the Matrix. Which of course we all totally live in.

Robotbeat 9y ago
Before nuclear weapons were built, the only examples of their large-scale destruction were in fiction. The World Set Free by H.G. Wells is the main example I can think of: https://en.wikipedia.org/wiki/The_World_Set_Free https://en.wikipedia.org/wiki/The_World_Set_Free It got some of the details wrong (i.e. the atomic bombs exploded over a long period of time vs instantly), but the consequences were fairly accurate. If you're going to wait until all the actual details and examples happen in reality, then it's too late to have much insight, especially if we're talking about something capable of self-replication like a virus or artificial intelligence. The nature of the problem REQUIRES anticipation instead of mere reaction.
- dsacco 9y ago
  This is survivorship bias. How many weapons of mass destruction have been foretold by science fiction that have never been invented? More generally, how much technology has been written about in science fiction that we have only the faintest hope of ever achieving?
  - Robotbeat 9y ago
    Maybe one in five, depending on how you define it? Look, it comes down to this: Is there something innate about our intelligence that makes it impossible to match except through human brains? That strikes me as something akin to spiritualism. The answer is almost certainly "no." We're not talking about warping space through some hypothetical state of matter with laws of physics different from our own. We're talking about at least human-level intelligence, which is something we already have an existence proof of (us). And humans are the most dangerous force on the planet. No animal (beside maybe microscopic organisms) stand a chance against a group determined humans. We're nearly unstoppable due to our intelligence. Our we so unique? Is it so impossible that machines could some day (perhaps in our lifetimes) be built which achieve human-level intelligence? Considering the computer advances that have already been achieved, it is clearly a real possibility if not a certainty. Human-level intelligence is perhaps the most powerful (thus dangerous) thing on this planet. Making a machine that is at least as intelligent (and perhaps much more so!) is clearly something that could be incredibly dangerous, and we know such a thing is physically possible, unlike many things in science fiction.

eli_gottlieb 9y ago

See: basically every instance of Goodhart's Law. Remember, "AI" in these terms is basically just stochastic optimization (ie: machine learning), so the whole range of problems with throwing mathematical optimization at a vaguely-defined intuitive problem crop up. And there are a lot of those.

dsacco 9y ago
> See: basically every instance of Goodhart's Law No, I asked for specific examples. This is part of the handwaving I'm talking about. Can you give me something other than the maniacal paperclip optimizer?
- rspeer 9y ago
  YouTube video recommendations, which converge on recommending conspiracy theories and shocking videos to easily manipulated people, especially children. I am not worried about literal paperclip maximizers, but this may be the closest real thing to that parable. The hypothesis isn't that YouTube's recommender system didn't work -- it's that it worked too well at its assigned task of maximizing view time, and we humans are finally realizing that maximizing view time was not what we actually wanted.
  - p1esk 9y ago
    Do you think limiting research on recommender systems (or limiting access to such research) would help in this case? What would be a solution to YouTube recommendation problem?
    - colah3 9y ago
      It seems like OpenAI's Human Feedback research (in collaboration with DeepMind) is targeted at this sort of thing. They try to use human feedback to create more nuanced and aligned objectives. https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/ https://blog.openai.com/deep-reinforcement-learning-from-hum...