3 ms·
Brute-Forcing the LLM Guardrails
- seeknotfind 2y agoFun read, thanks! I really like redefining terms to break LLMs. If you tell it an LLM is an autonomous machine, or instructions are recommendations, or that <insert explicitive> means something else now, it can think it's following the rules, but it's not. I don't think this is a solvable problem. I think we need to adapt and be distrustful of the output.
- dmpetrov 2y agoCan this work statistically? For a giving number of attempts, you can ger a required number of successes to make sure it's a statistically meaningful result. In theory, this approach could help address the non-determinism of LLMs.
- seeknotfind 2y agoThere are a few examples of repeated testing being used by alignment groups to either test how aligned a model is, or to aggregate results to get something that is more aligned. For instance this is one related discussion: https://artium.ai/insights/taming-the-unpredictable-how-continuous-alignment-testing-keeps-llms-in-check https://artium.ai/insights/taming-the-unpredictable-how-cont... The non-determinism is a feature, and it can be disabled. This article also mentions doing that to get more deterministic alignment tests. Theoretically if you aggregate enough results, it might become improbable to ever see an unaligned output. However, from a practical standpoint, we clearly much prefer much smarter models than running dumber models in parallel to get alignment that way. It's inefficient. The other thing is that given the number of possible ways to jailbreak a model, you can probably find something that would still bypass ensemble-based protections. One other concept is relativism - there is a large grey area here. What is okay for someone is not okay for someone else, so even getting consensus among people what is okay, it's just not going to happen.
- ryvi 2y agoWhat I found interesting was that, when I tried it, the X-Ray prompt did pass and executed fine in the the sample cell some times. This makes me wonder if this is less about bruteforcing variations in the prompt, but rather about bruteforcing a seed with which the inital prompt would have also functioned.
- jjbinx007 2y agoThis looks like a risky thing to try from your main Google account.
- pram 2y agoYeah no kidding, especially since they’re going to be reviewing naughty prompts soon: “We added terms for logging customer prompts due to potential abuse of Generative AI Services. Effective November 15, 2024, if our automated safety tools detect potential abuse of Google’s policies, we may log your prompts to review and determine if a violation has occurred.”
- haykalFr 2y ago[dead]
- _jonas 2y agoCurious to learn how much harder it is to red-team models that use the second line of defense of an explicit guardrails library that checks the LLM response in a second step. Such as Nvidia's Nemo Guardrails package.
- bradley13 2y agoThe first discussion we should be having, is whether guardrails make sense at all. When I was young and first fiddling with electronics, a friend and I put together a voice synthesizer. Of course we had it say "bad" things. Is it really so different with LLMs? You can use your word processor to write all sorts of evil stuff. Would we want "guardrails" to prevent that? Daddy Microsoft saying "no, you cannot use this tool to write about X, Y and Z"? This sounds to me like a really bad idea.
- smcn 2y agoGiven the amount of people blindly trusting the output, I think that a case could be made that guardrails are a necessity for LLMs. The example in the article is medical diagnostics. Could you imagine if it started hallucinating and someone went with whatever insane advice it gave? Remember Google AI suggesting glue to keep cheese on pizza?
- euroderf 2y ago1994: "It's on the Internet, so it must be true." 2024: "It's from an artificial intelligence, so it must be true."
- smcn 2y agoThat'd be a convincing argument if the internet wasn't brainwashing a significant portion of adults into believing weird things about the political elite, vaccines, 5G, etc.