6 ms·

It's also making sure AI is aligned with "our" intent and that "our" is a board made up of large corporations. If AI did run away and do it's own thing (seems

It's also making sure AI is aligned with "our" intent and that "our" is a board made up of large corporations. If AI did run away and do it's own thing (seems super unlikely) it's probably a crapshoot as to whether what it does is worse than the environmental apocalypse we live in where the rich continue to get richer and the poor poorer.

ben_w 2y ago
It can only be "super unlikely" for an AI to "run away and do it's own thing" when we actually know how to align it. Which we don't. So we're not aligning it with corporate boards yet, though not for lack of trying. (While LLMs are not directly agents, they are easy enough to turn into agents, and there's plenty of people willing to do that and disregard any concerns about the wisdom of this). So yes, the crapshoot is exactly what everyone in AI alignment is trying to prevent. (There's also, confusingly, "AI safety", which includes alignment but also covers things like misuse, social responsibility, and so on)
- root_axis 2y ago
  "Run away" AI is total science fiction - i.e, not anything happening in the foreseeable future. That's simply not how these systems work. Any looming AI threat will be entirely the result of deliberate human actions.
  - ben_w 2y ago
    We've already had robots "run away" into a water feature in one case and a pedestrian pushing a bike in another, the phrase doesn't only mean getting paperclipped. And for non-robotic AI, also flash-crashes on the stock market and that thing with Amazon book pricing bots caught up in a reactive cycle that drove up prices for a book they didn't have.
    - root_axis 2y ago
      > the phrase doesn't only mean getting paperclipped. This is what most people mean when they say "run away", i.e. the machine behaves in a surreptitious way to do things it was never designed to do, not a catastrophic failure that causes harm because the AI did not perform reliably.
      - ben_w 2y ago
        Every error is surreptitious to those who cannot predict the behaviour of a few billion matrix operations, which is most of us. When people are not paying attention, they're just as dead if it's Therac-25 or Thule airforce base early warning radar or an actual paperclipper.
        root_axis 2y ago
        No. Surreptitious means done with deliberate stealth to conceal your actions, not a miscalculation that results in a failure.
        ben_w 2y ago
        To those who are dead, that's a distinction without a difference. So far as I'm aware, none of the "killer robots gone wrong" actual sci-fi starts with someone deliberately aiming to wipe themselves out, it's always a misspecification or an unintended consequence. The fact that we don't know how to determine if there's a misspecification or an unintended consequence is the alignment problem.
        root_axis 2y ago
        "Unintended consequences" has nothing to do with AI specifically, it's a problem endemic to every human system. The "alignment" problem has no meaning in today's AI landscape beyond whether or not your LLM will emit slurs.
        8note 2y ago
        To those who are dead, it doesn't matter if there was a human behind the wheel, or a matrix

ben_w 2y ago

It can only be "super unlikely" for an AI to "run away and do it's own thing" when we actually know how to align it. Which we don't. So we're not aligning it with corporate boards yet, though not for lack of trying. (While LLMs are not directly agents, they are easy enough to turn into agents, and there's plenty of people willing to do that and disregard any concerns about the wisdom of this). So yes, the crapshoot is exactly what everyone in AI alignment is trying to prevent. (There's also, confusingly, "AI safety", which includes alignment but also covers things like misuse, social responsibility, and so on)

root_axis 2y ago
"Run away" AI is total science fiction - i.e, not anything happening in the foreseeable future. That's simply not how these systems work. Any looming AI threat will be entirely the result of deliberate human actions.
- ben_w 2y ago
  We've already had robots "run away" into a water feature in one case and a pedestrian pushing a bike in another, the phrase doesn't only mean getting paperclipped. And for non-robotic AI, also flash-crashes on the stock market and that thing with Amazon book pricing bots caught up in a reactive cycle that drove up prices for a book they didn't have.
  - root_axis 2y ago
    > the phrase doesn't only mean getting paperclipped. This is what most people mean when they say "run away", i.e. the machine behaves in a surreptitious way to do things it was never designed to do, not a catastrophic failure that causes harm because the AI did not perform reliably.
    - ben_w 2y ago
      Every error is surreptitious to those who cannot predict the behaviour of a few billion matrix operations, which is most of us. When people are not paying attention, they're just as dead if it's Therac-25 or Thule airforce base early warning radar or an actual paperclipper.
      - root_axis 2y ago
        No. Surreptitious means done with deliberate stealth to conceal your actions, not a miscalculation that results in a failure.
        ben_w 2y ago
        To those who are dead, that's a distinction without a difference. So far as I'm aware, none of the "killer robots gone wrong" actual sci-fi starts with someone deliberately aiming to wipe themselves out, it's always a misspecification or an unintended consequence. The fact that we don't know how to determine if there's a misspecification or an unintended consequence is the alignment problem.
        root_axis 2y ago
        "Unintended consequences" has nothing to do with AI specifically, it's a problem endemic to every human system. The "alignment" problem has no meaning in today's AI landscape beyond whether or not your LLM will emit slurs.
        8note 2y ago
        To those who are dead, it doesn't matter if there was a human behind the wheel, or a matrix