So again, the point of most security systems isn't to be impenetrable against dedicated attackers. They're supposed to stop relatively casual attackers and to

So again, the point of most security systems isn't to be impenetrable against dedicated attackers. They're supposed to stop relatively casual attackers and to establish that measures had been taken to secure the system. In the case of doorlocks having a simple Kwickset that the Lockpicking Lawyer could get through in 20 seconds is really sufficient security for most people. That keeps out the people rattling doorknobs looking for the person who left their house unlocked and a laptop on the coffee table. It also establishes that attempts had been made to secure the system. Similarly, the systems that block keywords in usernames on games don't have to be perfect either, some rate of both positive and negative failures are acceptable. The system works to block casual abuse, while users who go out of their way to circumvent the systems really establish the fact that they've actively worked around those systems, which makes the justification for punishment easier. And we pretty much know that in the case of prompt engineering that by disclosing the prompts that were used to secure the system would defeat the system and people would immediately publish how to work around the prompts. There isn't any use in independent auditing, because its a never ending cat and mouse game. And the failures of the system ARE NOT as significant as the failures in doorlocks or even keyword banning. False positives mean that you can't get what you want out of the system, which is just a failure in usability. It isn't like being locked out of your house or having your stuff stolen. And false negatives just means that the company has to work to improve the systems and whatever embarrassing content was constructed can be handled by PR. Since the company worked to prevent casual abuse and avoided a racist-tay-chatbot situation most people understand that going out of your way to hack prompts doesn't indicate that the company was negligent. And I don't see the parallels with the Kafkaesque systems that kick you out of systems which have turned into economic necessities like locking you out of your Google or GitHub or Apple accounts. All that is at stake here is that the prompt you wanted answered didn't work. That's just a usability problem.