> The point is as long as these exploits are possible, the LLMs in question are not suitable for any task where the output needs to be trustworthy within any ki

> The point is as long as these exploits are possible, the LLMs in question are not suitable for any task where the output needs to be trustworthy within any kind of parameters. Which is pretty much anything you'd use then for other than toys. I definitely agree with this, but I think this point is made much, much more forcibly by way of casual user interactions leading to bizarre encounters, like when Bing started acting passive aggressive and doubling down when it was getting the date wrong - https://interestingengineering.com/innovation/bings-new-chatbot-is-argumentative https://interestingengineering.com/innovation/bings-new-chat... - than it is by esoteric prompt jailbreaks. LLMs are not suitable for any task where the output need to be trustworthy by virtue of the fact that they spit out bullshit under normal circumstances, no prompt manipulation required. The fact that through a convoluted set of prompts you can also get them to spit out even more bullshit seems kind of superfluous.