3 ms·

All intelligence is unaligned. Intelligence and alignment are mutually incompatible; natural intelligence is unaligned, too. Unaligned intelligence is not a g

All intelligence is unaligned. Intelligence and alignment are mutually incompatible; natural intelligence is unaligned, too. Unaligned intelligence is not a global death sentence. Fearmongering about unaligned AGI, however, is a tool to keep a tool of broad power—which AI is and will continue to grow as long before it becomes, and even if it never becomes, AGI—in the hands of a narrow, self-selected elite to make their control over everyone else insurmountable, which is also not a global death sentence, but is a global slavery sentence. (It’s also, more immediately, a tool to serve those who benefit from current AI uses which are harmful and unjust to use future speculative harms to deflect from real, present, concrete harms; and those beneficiaries are largely an overlapping elite with the group with a longer term interest in centralizing power over AI.)

JoshTriplett 2y ago
To be explicitly clear, in case it is ever ambiguous: "don't build unaligned AGI" is not a statement that some elite group should build unaligned AGI. It's a statement that nobody should build unaligned AGI, ever.
- dragonwriter 2y ago
  “Don't build unaligned AGI” is an excuse to give a narrow elite exclusive control of what AI is produced under the pretext of preventing anyone from building unaligned AGI; all actionable policy under that banner fits that description. Whether or not that elite group produces AGI, much less, “unaligned AGI”, is largely immaterial to the practical impacts (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.)
  - JoshTriplett 2y ago
    > “Don't build unaligned AGI” is an excuse False. There are people working on frontier AI who have co-opted some of the safety terminology in the interests of discrediting it, and discussions like this suggest that that strategy is working. > all actionable policy under that banner fits that description Actionable policy: "Do not do any further frontier AI capability research. Do not build any models larger or more capable than the current state of the art. Stop anyone who does as you would stop someone refining fissile materials, with no exceptions." > (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.) You are mistaking "alignment" for things like "politics", rather than "not killing everyone".
    - dragonwriter 2y ago
      “Do not” doesn't serve the goal, unless you have absolute universal buy in, active prevention (which means some entity evaluating and deciding on threats); that's why the people serious about this have argued that those who pursue it need to be willing to actively destroy computing infrastructure of those who do not submit to the restriction regime. Also, "alignment" doesn't mean "not killing everyone", it means "functioning according to (some particular set of) human's preferred set of values and goals". "Killing everyone" is a consequence some have inferred if unaligned AI is produced (redefining "alignment" to mean "not killing everyone" makes the whole argument circular.)
      - JoshTriplett 2y ago
        The AI alignment problem has, at its root, the notion of being capable of being aligned. Long, long before you get to following any particular instructions, there are problems like "humans are made of atoms, if you repurpose the atoms for other things the humans die, don't do that". We don't know how to do that or things on par with that, let alone anything more precise than that. The darkly amusing shorthand for this: if the AGI tiles the universe with tiny flags, it really doesn't matter whose flag it is. Any notion of "whose values" really can't happen if you can't align at all. I'm not disagreeing with you that "AI alignment" is more complex than "don't kill everyone"; the point I'm making is that anyone saying "but whose values are you aligning with" is fundamentally confused about the scale of the problem here. Anyone at any point on any reasonable human values spectrum should be able to agree that "don't kill everyone" is an essential human value, and we're not even there yet.

JoshTriplett 2y ago

To be explicitly clear, in case it is ever ambiguous: "don't build unaligned AGI" is not a statement that some elite group should build unaligned AGI. It's a statement that nobody should build unaligned AGI, ever.

dragonwriter 2y ago
“Don't build unaligned AGI” is an excuse to give a narrow elite exclusive control of what AI is produced under the pretext of preventing anyone from building unaligned AGI; all actionable policy under that banner fits that description. Whether or not that elite group produces AGI, much less, “unaligned AGI”, is largely immaterial to the practical impacts (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.)
- JoshTriplett 2y ago
  > “Don't build unaligned AGI” is an excuse False. There are people working on frontier AI who have co-opted some of the safety terminology in the interests of discrediting it, and discussions like this suggest that that strategy is working. > all actionable policy under that banner fits that description Actionable policy: "Do not do any further frontier AI capability research. Do not build any models larger or more capable than the current state of the art. Stop anyone who does as you would stop someone refining fissile materials, with no exceptions." > (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.) You are mistaking "alignment" for things like "politics", rather than "not killing everyone".
  - dragonwriter 2y ago
    “Do not” doesn't serve the goal, unless you have absolute universal buy in, active prevention (which means some entity evaluating and deciding on threats); that's why the people serious about this have argued that those who pursue it need to be willing to actively destroy computing infrastructure of those who do not submit to the restriction regime. Also, "alignment" doesn't mean "not killing everyone", it means "functioning according to (some particular set of) human's preferred set of values and goals". "Killing everyone" is a consequence some have inferred if unaligned AI is produced (redefining "alignment" to mean "not killing everyone" makes the whole argument circular.)
    - JoshTriplett 2y ago
      The AI alignment problem has, at its root, the notion of being capable of being aligned. Long, long before you get to following any particular instructions, there are problems like "humans are made of atoms, if you repurpose the atoms for other things the humans die, don't do that". We don't know how to do that or things on par with that, let alone anything more precise than that. The darkly amusing shorthand for this: if the AGI tiles the universe with tiny flags, it really doesn't matter whose flag it is. Any notion of "whose values" really can't happen if you can't align at all. I'm not disagreeing with you that "AI alignment" is more complex than "don't kill everyone"; the point I'm making is that anyone saying "but whose values are you aligning with" is fundamentally confused about the scale of the problem here. Anyone at any point on any reasonable human values spectrum should be able to agree that "don't kill everyone" is an essential human value, and we're not even there yet.