Or equally, you wouldn't expect a word processor to refuse to print morally questionable material. The morals that leading models like ChatGPT are also aligned

Or equally, you wouldn't expect a word processor to refuse to print morally questionable material. The morals that leading models like ChatGPT are also aligned to a very American puritanism - ChatGPT will refuse to discuss sex, for example - and errs on the side of conservatism. I think it's a side effect of the hype around AI. If AI can destroy humanity we better make sure we can't do anything nasty with it!