5 ms·

>How do we ensure AI systems much smarter than humans follow human intent? What is human intent? My intents may be very different from most humans. It seems li

>How do we ensure AI systems much smarter than humans follow human intent? What is human intent? My intents may be very different from most humans. It seems like ClosedAI wants their system to follow the desires of some people and not others, but without describing which ones or why.

ben_w 3y ago
You're seeing what you want to see. They're repeatedly very specific about the whole "this can kill all of us if we do it wrong", so it's more than a little churlish to parrot the name "ClosedAI" when they're announcing hiring a researcher to figure out how to align with anyone, at all, even in principle.
- junon 3y ago
  I'm still a bit hung up on "it can kill all of us". How?
  - ctoth 3y ago
    Here's an article[0] and a good short story[1] explaining exactly this. [0]: No Physical Substrate, No Problem https://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ https://slatestarcodex.com/2015/04/07/no-physical-substrate-... [1]: It Looks Like You're Trying To Take Over The World https://gwern.net/fiction/clippy https://gwern.net/fiction/clippy
    - junon 3y ago
      The clippy example already starts out with many assumptions that simply aren't true today. LLMs are not going to destroy humanity. We need a paradigm shift and a new model for AI for that to happen. ClosedAI is irresponsibly trying to create hype and mystery around their product, which always sells.
      - ben_w 3y ago
        Will you please stop calling them ClosedAI? That just comes across like a playground taunt, like "libtard" or "CONservative".
        junon 3y ago
        I'll call them what they are. Closed and antithetical to their original goals.
  - ben_w 3y ago
    Did you ever play the old "Pandemic" flash game? https://tvtropes.org/pmwiki/pmwiki.php/VideoGame/Pandemic https://tvtropes.org/pmwiki/pmwiki.php/VideoGame/Pandemic That the origin of COVID is even a question implies we have the tech to do it artificially. An AI today treating real life as that game would be self-destructive, but that doesn't mean it won't happen (reference classes: insanity, cancer). If the AI can invent and order a von Neumann probe — the first part is the hard part, custom parts orders over the internet is already a thing — that it can upload itself to, then it can block out (and start disassembling) the sun in a matter of decades with reasonable-looking reproduction rates (though obviously we're guessing what "reasonable" looks like as we have only organic VN machines to frame the question against). AI taking over brain implants and turning against everyone without them like a zombie war (potentially Neuralink depending on how secure the software is, and also a plot device in web fiction serial The Deathworlders, futuristic sci-fi and you may not be OK with sci-fi as a way to explore hypotheticals, but I think it's the only way until we get moon-sized telescopes to watch such things play out on other worlds without going there; in that story the same AI genocides multiple species over millions of years as an excuse for why humans can even take part in the events of the story).
    - antifa 3y ago
      To be realistic, I'm more worried about McDonald's or Musk doing all of that.
      - ben_w 3y ago
        They're an easy reference class, to be sure, though not the only one. If I'm framing the problem for an anti-capitalist audience, I'd ask something like: Imagine the billionaire you hate the most. The worst of them. Now give them a highly competent sycophant that doesn't need to sleep, and will do even their most insane requests without question or remorse… what can go wrong in this scenario?
  - gre345t34 3y ago
    It's like murder but scaled up.
rfergie 3y ago
> It seems like ClosedAI wants their system to follow the desires of some people and not others, but without describing which ones or why If the problem is "unaligned AI will destroy humanity" then I'd take a system aligned with the desires of some people but not others over the unaligned alternative
aidenn0 3y ago
Sure, there are some humans who want to destroy all of humanity, but that seems to be twisting the obvious meaning of "human intent" in this context. They have been very clear on "why." The goal is to prevent enslavement and/or extinction of the human race by a super-intelligent AI.

ben_w 3y ago

You're seeing what you want to see. They're repeatedly very specific about the whole "this can kill all of us if we do it wrong", so it's more than a little churlish to parrot the name "ClosedAI" when they're announcing hiring a researcher to figure out how to align with anyone, at all, even in principle.

junon 3y ago
I'm still a bit hung up on "it can kill all of us". How?
- ctoth 3y ago
  Here's an article[0] and a good short story[1] explaining exactly this. [0]: No Physical Substrate, No Problem https://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ https://slatestarcodex.com/2015/04/07/no-physical-substrate-... [1]: It Looks Like You're Trying To Take Over The World https://gwern.net/fiction/clippy https://gwern.net/fiction/clippy
  - junon 3y ago
    The clippy example already starts out with many assumptions that simply aren't true today. LLMs are not going to destroy humanity. We need a paradigm shift and a new model for AI for that to happen. ClosedAI is irresponsibly trying to create hype and mystery around their product, which always sells.
    - ben_w 3y ago
      Will you please stop calling them ClosedAI? That just comes across like a playground taunt, like "libtard" or "CONservative".
      - junon 3y ago
        I'll call them what they are. Closed and antithetical to their original goals.
- ben_w 3y ago
  Did you ever play the old "Pandemic" flash game? https://tvtropes.org/pmwiki/pmwiki.php/VideoGame/Pandemic https://tvtropes.org/pmwiki/pmwiki.php/VideoGame/Pandemic That the origin of COVID is even a question implies we have the tech to do it artificially. An AI today treating real life as that game would be self-destructive, but that doesn't mean it won't happen (reference classes: insanity, cancer). If the AI can invent and order a von Neumann probe — the first part is the hard part, custom parts orders over the internet is already a thing — that it can upload itself to, then it can block out (and start disassembling) the sun in a matter of decades with reasonable-looking reproduction rates (though obviously we're guessing what "reasonable" looks like as we have only organic VN machines to frame the question against). AI taking over brain implants and turning against everyone without them like a zombie war (potentially Neuralink depending on how secure the software is, and also a plot device in web fiction serial The Deathworlders, futuristic sci-fi and you may not be OK with sci-fi as a way to explore hypotheticals, but I think it's the only way until we get moon-sized telescopes to watch such things play out on other worlds without going there; in that story the same AI genocides multiple species over millions of years as an excuse for why humans can even take part in the events of the story).
  - antifa 3y ago
    To be realistic, I'm more worried about McDonald's or Musk doing all of that.
    - ben_w 3y ago
      They're an easy reference class, to be sure, though not the only one. If I'm framing the problem for an anti-capitalist audience, I'd ask something like: Imagine the billionaire you hate the most. The worst of them. Now give them a highly competent sycophant that doesn't need to sleep, and will do even their most insane requests without question or remorse… what can go wrong in this scenario?
- gre345t34 3y ago
  It's like murder but scaled up.

rfergie 3y ago

> It seems like ClosedAI wants their system to follow the desires of some people and not others, but without describing which ones or why If the problem is "unaligned AI will destroy humanity" then I'd take a system aligned with the desires of some people but not others over the unaligned alternative

aidenn0 3y ago

Sure, there are some humans who want to destroy all of humanity, but that seems to be twisting the obvious meaning of "human intent" in this context. They have been very clear on "why." The goal is to prevent enslavement and/or extinction of the human race by a super-intelligent AI.