3 ms·

Humans do not agree on what values are correct, but values can be averaged. So for example if a family with 5 children is on vacation, do you maintain that it

Humans do not agree on what values are correct, but values can be averaged. So for example if a family with 5 children is on vacation, do you maintain that it is impossible even in principle for the parents to take the preferences of all 5 children into account in approximately equal measure as to what activities or non-activities to pursue? Also: are you pursuing a complete tangent or do you see your point as bearing on whether frontier AI research should be banned? (If so, I cannot tell whether you consider your point to support a ban or oppose a ban.)

janalsncm 2y ago
The vast majority of harms from “AI” are actually harms from the corporations and governments that control them, who have mutually incompatible goals, getting what they want. This is why alignment folks at OpenAI are quickly learning that the first problem they need to solve is what happens when their values don’t align with the company’s (spoiler: they get fired). Therefore the actual solution is not coming up with more and more clever “guardrails” but aligning corporations and governments to human needs. In other words, politics. There are other problems like enabling new types of scams which will require political solutions. At a technical level the best these companies can do is mitigation.
- JoshTriplett 2y ago
  > The vast majority of harms from “AI” Don't extrapolate from present harms to future harms, here. The problem AI alignment is trying to solve at a most basic level is "don't kill everyone", and even that much isn't solved yet. Solving that (or, rather, buying time to solve it) will require political solutions, in the sense of international diplomacy. But it has absolutely nothing to do with "aligning corporations", and everything to do with teaching computers things on par with (oversimplifying here) "humans are made up of atoms, and if you repurpose those atoms the humans die, don't ever do that".
  - dragonwriter 2y ago
    > The problem AI alignment is trying to solve is "don't kill everyone". No, its not. AI alignment was an active area of concern (and the fundamental problem for useful AI with significant autonomy) before cultists started trying to reduce the scope of its problem space from the wide scope of real problems it concerns to a single speculative apocalypse.
    - hollerith 2y ago
      No, what actually happened is that the people you are calling the cultists coined the term alignment, which then got appropriated by the AI labs. But the genesis of the term "alignment" (as applied to AI) is a side issue. What is important is that reinforcement learning with human feedback and the other techniques used on the current crop of AIs to make it less likely that the AI will say things that embarass the owner of the AI are fundamentally different from making sure the an AI that turns out more capable than us will not kill us all or do something else awful.
      - dragonwriter 2y ago
        That's simply factually untrue, and even some of the people who have become apocalypse cultists used "alignment" in the original sense before coming to advocate apocalypse as the only issue of concern.
        comp_throw7 2y ago
        The term "alignment" was first coined in this context as a suggestion from Stuart Russell to MIRI, in 2014: https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-the-term-ai-alignment-coined?commentId=kFahR6wLZr35krNZ5 https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-t... Both, of course, are concerned primarily with the risk of human extinction from AI.
  - janalsncm 2y ago
    > The problem AI alignment is trying to solve at a most basic level is "don't kill everyone", and even that much isn't solved yet The fact that the number of things that could hypothetically lead to human extinction is entirely unbounded and (since we’re not extrapolating from present harms) unpredictable is a very convenient fact for people who are paid for their time in “solving” this problem.
    - deleted 2y ago
      [deleted]

janalsncm 2y ago

The vast majority of harms from “AI” are actually harms from the corporations and governments that control them, who have mutually incompatible goals, getting what they want. This is why alignment folks at OpenAI are quickly learning that the first problem they need to solve is what happens when their values don’t align with the company’s (spoiler: they get fired). Therefore the actual solution is not coming up with more and more clever “guardrails” but aligning corporations and governments to human needs. In other words, politics. There are other problems like enabling new types of scams which will require political solutions. At a technical level the best these companies can do is mitigation.

JoshTriplett 2y ago
> The vast majority of harms from “AI” Don't extrapolate from present harms to future harms, here. The problem AI alignment is trying to solve at a most basic level is "don't kill everyone", and even that much isn't solved yet. Solving that (or, rather, buying time to solve it) will require political solutions, in the sense of international diplomacy. But it has absolutely nothing to do with "aligning corporations", and everything to do with teaching computers things on par with (oversimplifying here) "humans are made up of atoms, and if you repurpose those atoms the humans die, don't ever do that".
- dragonwriter 2y ago
  > The problem AI alignment is trying to solve is "don't kill everyone". No, its not. AI alignment was an active area of concern (and the fundamental problem for useful AI with significant autonomy) before cultists started trying to reduce the scope of its problem space from the wide scope of real problems it concerns to a single speculative apocalypse.
  - hollerith 2y ago
    No, what actually happened is that the people you are calling the cultists coined the term alignment, which then got appropriated by the AI labs. But the genesis of the term "alignment" (as applied to AI) is a side issue. What is important is that reinforcement learning with human feedback and the other techniques used on the current crop of AIs to make it less likely that the AI will say things that embarass the owner of the AI are fundamentally different from making sure the an AI that turns out more capable than us will not kill us all or do something else awful.
    - dragonwriter 2y ago
      That's simply factually untrue, and even some of the people who have become apocalypse cultists used "alignment" in the original sense before coming to advocate apocalypse as the only issue of concern.
      - comp_throw7 2y ago
        The term "alignment" was first coined in this context as a suggestion from Stuart Russell to MIRI, in 2014: https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-the-term-ai-alignment-coined?commentId=kFahR6wLZr35krNZ5 https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-t... Both, of course, are concerned primarily with the risk of human extinction from AI.
- janalsncm 2y ago
  > The problem AI alignment is trying to solve at a most basic level is "don't kill everyone", and even that much isn't solved yet The fact that the number of things that could hypothetically lead to human extinction is entirely unbounded and (since we’re not extrapolating from present harms) unpredictable is a very convenient fact for people who are paid for their time in “solving” this problem.
  - deleted 2y ago
    [deleted]