This is about the ‘political, legal, and PR protection’ kind of alignment to avoid news stories about kids making bombs thanks to an enthusiastically accommodat

This is about the ‘political, legal, and PR protection’ kind of alignment to avoid news stories about kids making bombs thanks to an enthusiastically accommodating GPT. Language models are set to be the new encyclopedias— what a model presents as real is going to be absorbed as real by many people. Considering the implications of that isn’t an issue of emotional oversensitivity. Further, there’s a case for a public facing chat assistant to be neutral about anything other than upholding the status quo. Do you want to trust the opinion of a chatbot as for when and who against an armed uprising is appropriate? This is not really about a threat model of AGI turning our world into a dystopia or paperclips. However, your disdain for people who are being thoughtful about the future and ‘thought experiments’ seems brash and unfounded. Thought experiments have been incredibly useful throughout history and are behind things like the theory of relativity. Nuclear stalemate via mutually assured destruction is a ‘thought experiment,’ and one I’m not eager to see validated outside of thought.