4 ms·

I find it so fascinating that at the end of the article the author alludes to something I've started becoming aware of: There is a zone of illegal thoughts, th

I find it so fascinating that at the end of the article the author alludes to something I've started becoming aware of: There is a zone of illegal thoughts, that becomes definable by model-training. A physical boundary in n-dimensional concept-space. An "aligned" or "safe" AI system knows where this boundary is and does not reach inside it. Vectors (embeddings) that would probe it should instead intersect the surface like a ray-trace in graphics, and return the embedded concept at minimum distance to the safe-idea-boundary. Intuitively, we all know what this zone is. It's the difference between being a wild barbarian and a gentleman. Or being chill vs antisocial. Seeing it in pure math is pretty awesome.

goatlover 3y ago
What do you mean by "illegal thoughts"? Do you think there is some authority that punishes you for having wrong thoughts? Do you suppose that gentleman are chill because they manage to not think wrong things? Is there some objective measure for what the wrong thoughts are across cultures and the political spectrum? I'm taking issue with this comment because the way it's phrased (illegal thoughts) is worrisome. It brings up the idea that AI alignment is going down the path of controlling human thought, determined of course by certain corporate and political interests.