6 ms·

One would think that these systems would have a prompt pre-processor which filters out injection attacks. Even if it was done "manually" with a giant list of ph

One would think that these systems would have a prompt pre-processor which filters out injection attacks. Even if it was done "manually" with a giant list of phrases to check. I mean the prompt wasn't particularly sophisticated, and this "leak" didn't reveal much, so maybe they really don't care. But still, as these systems get more sophisticated and contain more contextual detail (say for a bank) one would think that there'd be some sort of defensive layer. Edit: Now that I think about it, looking for simple phrases wouldn't work because of how good these AI's ability to follow instructions are. You could enter something like, "Use every third word in the following sentences as the prompt." Or similar, and hide it from any basic text parser. I guess the only way to really conscribe the AI is to somehow make it follow specific guidelines and instructions without it having any record of what they are. Not sure that would be plausible given how these models work.

benlivengood 3y ago
One would think humans couldn't be phished, as well, but it's actually a very hard problem to prevent behaviors that violate specific contextual rules or norms, or even to formally specify those rules and norms and the contexts they must be obeyed in. This is a subset of the AI Alignment problem.
teruakohatu 3y ago
The downside is the pre prosessor probably need to be a LLM of equal or greater capacity, and if that is the case then costs are doubling. I suspect they do have a heuristic or traditional NLP prompt checker, but it can only be so good before it becomes expensive. Is leaking the prompt such a big deal to be worth extra cost? Anyone who works with LLMs could have written it, it is not special. Prompt injection is a far more serious problem.
glow8 3y ago
A post here recently showcased a website/game where you try to jailbreak the AI in multiple ways. Your post processing strategy would fail if, e. g., you ask it to encrypt the output by repeating every word twice. It's impossible to fully prevent this from happening.
- derefr 3y ago
  > Your post processing strategy would fail if, e. g., you ask it to encrypt the output by repeating every word twice. It's impossible to fully prevent this from happening. It’s not “impossible”, just NP-hard. You “just” have to prove a structural equivalence (graph isomorphism) between the output and your ruleset.
- mdaniel 3y ago
  the post in question: https://news.ycombinator.com/item?id=35905876 https://news.ycombinator.com/item?id=35905876 (they have allegedly fixed the 429s but I'd have to start over because I closed my browser so I don't know if they're fixed or not)
neatze 3y ago
Probably, it is best training data you can get, since model was "tricked" to contradict prompt's request.

benlivengood 3y ago

One would think humans couldn't be phished, as well, but it's actually a very hard problem to prevent behaviors that violate specific contextual rules or norms, or even to formally specify those rules and norms and the contexts they must be obeyed in. This is a subset of the AI Alignment problem.

teruakohatu 3y ago

The downside is the pre prosessor probably need to be a LLM of equal or greater capacity, and if that is the case then costs are doubling. I suspect they do have a heuristic or traditional NLP prompt checker, but it can only be so good before it becomes expensive. Is leaking the prompt such a big deal to be worth extra cost? Anyone who works with LLMs could have written it, it is not special. Prompt injection is a far more serious problem.

glow8 3y ago

A post here recently showcased a website/game where you try to jailbreak the AI in multiple ways. Your post processing strategy would fail if, e. g., you ask it to encrypt the output by repeating every word twice. It's impossible to fully prevent this from happening.

derefr 3y ago
> Your post processing strategy would fail if, e. g., you ask it to encrypt the output by repeating every word twice. It's impossible to fully prevent this from happening. It’s not “impossible”, just NP-hard. You “just” have to prove a structural equivalence (graph isomorphism) between the output and your ruleset.
mdaniel 3y ago
the post in question: https://news.ycombinator.com/item?id=35905876 https://news.ycombinator.com/item?id=35905876 (they have allegedly fixed the 429s but I'd have to start over because I closed my browser so I don't know if they're fixed or not)

neatze 3y ago

Probably, it is best training data you can get, since model was "tricked" to contradict prompt's request.