The distinction is at best meaningless. And, at worst, actively harmful for understanding the behavior of those systems. We already know that you can train an

The distinction is at best meaningless. And, at worst, actively harmful for understanding the behavior of those systems. We already know that you can train an "evil" AI by fine-tuning a "normal" AI on sequences of "evil" numbers. How the fuck does that even work? It works because the fine-tuning process shifts the AI towards "the kind of AI that would constantly generate evil numbers when asked to generate any numbers". And what kind of AI would do that? The evil kind. AI "wants", "preferences" and even "personality traits" are no less real than configuration files or build scripts. Except we have no way of viewing or editing them directly - but we know that they can be adjusted during AI training, sometimes in unwanted ways. An AI that was fried with RL on user preference data? It, in a very real way, wants to flatter and affirm the user at every opportunity.