5 ms·

"Superalignment" is (was?) OpenAI's term, so it might be more accurate to say he is joining Anthropic to work on alignment.

eminence32 2y ago
Is there a difference between "superalignment" and "alignment" ?
- exe34 2y ago
  i suppose the difference between imaginary and "super"-imaginary isn't very important from a practical point of view. they worry about alignment for ai, I worry about alignment for the corporations that wield technology, any technology.
- rfw300 2y ago
  Yes. “Superalignment” (admittedly a corny term) refers to the specific case of aligning AI systems that are more intelligent than human beings. Alignment is an umbrella term which can also refer to basic work like fine-tuning an LLM to follow instructions.
  - afefers 2y ago
    Huh! All this time I thought the "super" was just for branding/differentiation.
    - deleted 2y ago
      [deleted]
    - zucker42 2y ago
      That was definitely part of it.
    - cwillu 2y ago
      Alignment was the original term, but has been largely coopted to mean a vaguely similar looking concept of public safety around the capabilities of current models.
  - thefaux 2y ago
    Is this not something of an oxymoron? If there exists an ai that is more intelligent than humans, how could we mere mortals hope to control it? If we hinder it so that it cannot act in ways that harm humans, can we really be said to have created superintelligence? It seems to me that the only way to achieve superalignment is to not create superintelligence, if that is even within our control.
    - whimsicalism 2y ago
      Many people share your views, but others believe it is possible.
    - renewiltord 2y ago
      Not self-evident. Fungus can control ant. Toxoplasma gondii can control human. Who is more intelligent? So if control of more intelligent being is possible, could it be symbiotic to permit? Alpha-proteobacteria sister to ancestor proto-mitochondria and now we live aligned. But those beings lacked conscious agency. We have more than them. Not self-evident we will fail at this.
      - dr_dshiv 2y ago
        Another example is the alignment between our hindbrain, limbic system and neocortex. Neocortex is smarter but is usually controlled by lower level processes… Note that misalignment between these systems is very common.
  - halfjoking 2y ago
    Then why don't they call politicians "super-politicians"? Their purpose is to control the population by being lesser beings who feed off corporations and just push their message.
- deleted 2y ago
  [deleted]
- throw5345346 2y ago
  Oh yes. One is super.
- deleted 2y ago
  [deleted]
sp332 2y ago
Looks like superalignment was Jan Leike's term, since the team at OpenAI dissolved immediately without him.

eminence32 2y ago

Is there a difference between "superalignment" and "alignment" ?

exe34 2y ago
i suppose the difference between imaginary and "super"-imaginary isn't very important from a practical point of view. they worry about alignment for ai, I worry about alignment for the corporations that wield technology, any technology.
rfw300 2y ago
Yes. “Superalignment” (admittedly a corny term) refers to the specific case of aligning AI systems that are more intelligent than human beings. Alignment is an umbrella term which can also refer to basic work like fine-tuning an LLM to follow instructions.
- afefers 2y ago
  Huh! All this time I thought the "super" was just for branding/differentiation.
  - deleted 2y ago
    [deleted]
  - zucker42 2y ago
    That was definitely part of it.
  - cwillu 2y ago
    Alignment was the original term, but has been largely coopted to mean a vaguely similar looking concept of public safety around the capabilities of current models.
- thefaux 2y ago
  Is this not something of an oxymoron? If there exists an ai that is more intelligent than humans, how could we mere mortals hope to control it? If we hinder it so that it cannot act in ways that harm humans, can we really be said to have created superintelligence? It seems to me that the only way to achieve superalignment is to not create superintelligence, if that is even within our control.
  - whimsicalism 2y ago
    Many people share your views, but others believe it is possible.
  - renewiltord 2y ago
    Not self-evident. Fungus can control ant. Toxoplasma gondii can control human. Who is more intelligent? So if control of more intelligent being is possible, could it be symbiotic to permit? Alpha-proteobacteria sister to ancestor proto-mitochondria and now we live aligned. But those beings lacked conscious agency. We have more than them. Not self-evident we will fail at this.
    - dr_dshiv 2y ago
      Another example is the alignment between our hindbrain, limbic system and neocortex. Neocortex is smarter but is usually controlled by lower level processes… Note that misalignment between these systems is very common.
- halfjoking 2y ago
  Then why don't they call politicians "super-politicians"? Their purpose is to control the population by being lesser beings who feed off corporations and just push their message.
deleted 2y ago
[deleted]
throw5345346 2y ago
Oh yes. One is super.
deleted 2y ago
[deleted]

sp332 2y ago

Looks like superalignment was Jan Leike's term, since the team at OpenAI dissolved immediately without him.