4 ms·

> The hardest issue regarding "AI alignment" is human alignment. Which is partly why the current proposed alignment theory isn't possible. We want to align the

> The hardest issue regarding "AI alignment" is human alignment. Which is partly why the current proposed alignment theory isn't possible. We want to align the AGI by applying human values. Even if we figure out how to get the machine to adopt such values, they are the same values that lead us humans into constant conflict. I've stated this argument in much more detail here - https://dakara.substack.com/p/ai-singularity-the-hubris-trap https://dakara.substack.com/p/ai-singularity-the-hubris-trap

Verdex 4y ago
With all the AI discussion of late I've been thinking more about the alignment problem. Like, let's say you have a true AGI that's completely superior to human intellect AND you've found a way to align this thing. Can those alignment techniques also be used on the lesser mind of humanity? "Oh don't worry. Alignment is probably achievable by how we build the AGI." Yeah I mean maybe. Or maybe it turns out alignment is only possible at all if it works generally on anything with 'intelligence'. My suspicion is that alignment is either not possible because an intelligent agent can always do something just because. Or we're going to have to worry about whoever wins the alignment race aligning the rest of us.
- 13years 4y ago
  Yes, I think those are possibilities. In simple terms, I think we will either be in control of power we are not prepared to manage or we will be managed by power that we can not control.
ChatGTP 4y ago
I think you’re a brilliant writer. This is a fantastic article. Thank you for articulating the problem so clearly. The only hope I have is that because we’re building these things largely in our image, because we’re fallible and often the systems we build are too, we’re actually building something quite stupid and incapable of mass scale destruction. Hopefully the laws of physics or intellect are reached before anything gets to too out of control. This is why “containment” is such a precarious concept. In the end, we may find our plan to place the AI into checkmate only resulted in placing ourselves into checkmate.
- 13years 4y ago
  Thank you! > because we’re fallible and often the systems we build are too Yes, the basis of my article is using the assumptions of the alignment theorists who believe that we can build an AGI and it will be by default "power seeking". Of course, either could be wrong. I agree that is really the best hope that either we hit some unexpected limitations or the problem we perceive simply does not manifest as we fear.
  - ChatGTP 4y ago
    I suppose we’re always living on the edge, this time is no different.

Verdex 4y ago

With all the AI discussion of late I've been thinking more about the alignment problem. Like, let's say you have a true AGI that's completely superior to human intellect AND you've found a way to align this thing. Can those alignment techniques also be used on the lesser mind of humanity? "Oh don't worry. Alignment is probably achievable by how we build the AGI." Yeah I mean maybe. Or maybe it turns out alignment is only possible at all if it works generally on anything with 'intelligence'. My suspicion is that alignment is either not possible because an intelligent agent can always do something just because. Or we're going to have to worry about whoever wins the alignment race aligning the rest of us.

13years 4y ago
Yes, I think those are possibilities. In simple terms, I think we will either be in control of power we are not prepared to manage or we will be managed by power that we can not control.

ChatGTP 4y ago

I think you’re a brilliant writer. This is a fantastic article. Thank you for articulating the problem so clearly. The only hope I have is that because we’re building these things largely in our image, because we’re fallible and often the systems we build are too, we’re actually building something quite stupid and incapable of mass scale destruction. Hopefully the laws of physics or intellect are reached before anything gets to too out of control. This is why “containment” is such a precarious concept. In the end, we may find our plan to place the AI into checkmate only resulted in placing ourselves into checkmate.

13years 4y ago
Thank you! > because we’re fallible and often the systems we build are too Yes, the basis of my article is using the assumptions of the alignment theorists who believe that we can build an AGI and it will be by default "power seeking". Of course, either could be wrong. I agree that is really the best hope that either we hit some unexpected limitations or the problem we perceive simply does not manifest as we fear.
- ChatGTP 4y ago
  I suppose we’re always living on the edge, this time is no different.