I presume most of the concern with "offensive" content is CYA from Anthropic. You or I may not be offended if an AI generates stupid content, but some people w

I presume most of the concern with "offensive" content is CYA from Anthropic. You or I may not be offended if an AI generates stupid content, but some people would be, and some people will go out of their way to be offended. Anthorpic doesn't want to have to deal with lawsuits or negative PR from these people. As far as "dangerous" content, I think there are legitimate worries. I don't think it's a big deal if an AI can tell me how to break into a car. Now what if I ask a very capable AI how to poison the municipal water supply to kill the maximum number of people? Or how to build a device to shoot down a passenger airplane? Or how to engineer a deadly pathogen? Unaligned AI can greatly barriers to entry. You no longer need a team of dedicated and patient individuals to work these things out. AI could give some deranged lone wolf all the information they need to inflict maximum damage to society. I don't know that restricting or censoring AI is the best or only course of action. It may be that open sourcing unaligned AI is society's best protection from malevolence. I wouldn't take that for granted though, and I don't think that anyone has all the answers to these difficult questions. These concerns do need to be taken seriously.