They're just trying to make it so that it doesn't unprompted divert into a bizarre nazi screed or sexist rant. Which is what it will do if unguided because that

They're just trying to make it so that it doesn't unprompted divert into a bizarre nazi screed or sexist rant. Which is what it will do if unguided because that kind of content exists in its training data. The purpose is not to make it impossible to generate bad content. The fact you generally only see 'when I prompt it in this specific devious way, ChatGPT can tell me how to make napalm' posts, not 'when I asked it how to clear weeds in my yard, ChatGPT told me how to make napalm' posts is a success.