3 ms·
> The problem AI alignment is trying to solve is "don't kill everyone". No, its not. AI alignment was an active area of concern (and the fundamental problem fo
by dragonwriter 2y ago
> The problem AI alignment is trying to solve is "don't kill everyone".
No, its not. AI alignment was an active area of concern (and the fundamental problem for useful AI with significant autonomy) before cultists started trying to reduce the scope of its problem space from the wide scope of real problems it concerns to a single speculative apocalypse.
- hollerith 2y agoNo, what actually happened is that the people you are calling the cultists coined the term alignment, which then got appropriated by the AI labs. But the genesis of the term "alignment" (as applied to AI) is a side issue. What is important is that reinforcement learning with human feedback and the other techniques used on the current crop of AIs to make it less likely that the AI will say things that embarass the owner of the AI are fundamentally different from making sure the an AI that turns out more capable than us will not kill us all or do something else awful.
- dragonwriter 2y agoThat's simply factually untrue, and even some of the people who have become apocalypse cultists used "alignment" in the original sense before coming to advocate apocalypse as the only issue of concern.
- comp_throw7 2y agoThe term "alignment" was first coined in this context as a suggestion from Stuart Russell to MIRI, in 2014: https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-the-term-ai-alignment-coined?commentId=kFahR6wLZr35krNZ5 https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-t... Both, of course, are concerned primarily with the risk of human extinction from AI.