3 ms·

The concept is called egregore, and yes, any “AI alignment” discussion I read blissfully ignores that we have been unable to align neither states nor corporatio

The concept is called egregore, and yes, any “AI alignment” discussion I read blissfully ignores that we have been unable to align neither states nor corporations with human goals, while both are much dumber egregores than AI.

pavlov 2y ago
I would argue that today’s states and corporations are much more aligned with human goals than their equivalents from, say, 500 years ago. I’ll much rather have the Federal Republic of Germany and Google than Emperor Charles V and the Inquisition. Who’s to say that we can’t make similar progress in the next 500 years too?
- MichaelZuo 2y ago
  Why does the alignment relative to a prior point matter? e.g. A small snowball could be nearly perfectly enmeshed with the surrounding snow on top of a steep hill but that doesn’t stop the small snowball from rolling down the hill and becoming a very large snowball in a few seconds, and wrecking some unfortunate passer-by at the bottom. A few microns of freezing rain may have been the deciding factor so even a 99.9% relative ‘alignment’ between snowball and snowy hill top would still be irrelevant for the unlucky person. Who may have walked by 10000 times prior.
- OgsyedIE 2y ago
  Isn't this perceived alignment a mere instrumental goal carried out in the short-term?
- ben_w 2y ago
  "Alignment" can be taken literally, cosine similarity of the vectors <what you want> and <what the system does>. The more powerful the system is compared to you, the more any small difference is amplified. AI that's about as powerful as an intern, great, no big deal for us. AI that's capable enough to run a company? If it's morally "good" then great; if not trade unions and strikes are a thing, as are "union busters". AI that's capable enough to run a country? If it's morally "good" then great; if not…
ben_w 2y ago
I see the difficulty of aligning corporations being mentioned a few times, and I've brought it up also. Between SLAPP cases, FUD, lobbying, and the way all the harms occur despite being made out of humans, there's already a bunch of non-AI ways for powerful entities that harm us to make it difficult to organise ourselves against the harm.

pavlov 2y ago

I would argue that today’s states and corporations are much more aligned with human goals than their equivalents from, say, 500 years ago. I’ll much rather have the Federal Republic of Germany and Google than Emperor Charles V and the Inquisition. Who’s to say that we can’t make similar progress in the next 500 years too?

MichaelZuo 2y ago
Why does the alignment relative to a prior point matter? e.g. A small snowball could be nearly perfectly enmeshed with the surrounding snow on top of a steep hill but that doesn’t stop the small snowball from rolling down the hill and becoming a very large snowball in a few seconds, and wrecking some unfortunate passer-by at the bottom. A few microns of freezing rain may have been the deciding factor so even a 99.9% relative ‘alignment’ between snowball and snowy hill top would still be irrelevant for the unlucky person. Who may have walked by 10000 times prior.
OgsyedIE 2y ago
Isn't this perceived alignment a mere instrumental goal carried out in the short-term?
ben_w 2y ago
"Alignment" can be taken literally, cosine similarity of the vectors <what you want> and <what the system does>. The more powerful the system is compared to you, the more any small difference is amplified. AI that's about as powerful as an intern, great, no big deal for us. AI that's capable enough to run a company? If it's morally "good" then great; if not trade unions and strikes are a thing, as are "union busters". AI that's capable enough to run a country? If it's morally "good" then great; if not…

ben_w 2y ago

I see the difficulty of aligning corporations being mentioned a few times, and I've brought it up also. Between SLAPP cases, FUD, lobbying, and the way all the harms occur despite being made out of humans, there's already a bunch of non-AI ways for powerful entities that harm us to make it difficult to organise ourselves against the harm.