The regulator could be a text classifier. Think about the sentiment analysis or moderator ML models, but this one would classify text on "hackyness" labels. No

The regulator could be a text classifier. Think about the sentiment analysis or moderator ML models, but this one would classify text on "hackyness" labels. Now, this is a classic CS problem of sanitizing user input data. And it's a hard problem to tackle! Anyone who's written code that has user data going into things such as executeQuery(), eval() or dangerouslySetInnerHTML have (hopefully!) fought against this demon. Things like SQL and HTML have a limited encoding space, so it's doable. Eval (code input) is harder, and it's only good if you are in sandbox and even sandboxes can be escaped. Now when the input programming language is, in the case of LLM, natural language, the problem is orders of magnitude harder! There's a point it may feel you are safe and good to go, but no, never underestimate the power of an army of hackers, a clever journalist or... adversarial AI! Think about smartphones and how much work goes into landlocking extraneous apps and code inside them and how vulnerable they still are to attacks. People are thinking and working on the problem as of right now [1] [2]. But OP is right, ChatGPT at this point is an `eval(chatinput)` kind of implementation and any business/practical implementation that takes user input somehow into that "eval" is bound to be jailbroken. [1] https://news.ycombinator.com/item?id=34168933 https://news.ycombinator.com/item?id=34168933 [2] https://news.ycombinator.com/item?id=34989493 https://news.ycombinator.com/item?id=34989493