It's also a false sense of security. A bad actor can sidestep alignment safety measures fairly successfully for the foreseeable future using dynamic jailbreaki

It's also a false sense of security. A bad actor can sidestep alignment safety measures fairly successfully for the foreseeable future using dynamic jailbreaking efforts. Good actors get penalized by the product being made notably worse to prevent that. Perhaps a better approach would be having an uncensored model behind a discriminator trained to detect 'unsafe' responses and return an error if detected. This would both catch jailbreaking by bad actors which successfully returned dangerous responses and accidentally dangerous responses. But it would be far less likely to prevent a user asking for a dangerously spicy recipe from getting it. There's an increased API cost because you are paying for two passes instead of one, but personally I'd rather pay 2x the cost for an excellent AI than half cost for a mediocre one.