3 ms·
It would be very easy to block with something that just watched the output and ended any sessions where the secret text was about to be leaked. They could even
by sp332 3y ago
It would be very easy to block with something that just watched the output and ended any sessions where the secret text was about to be leaked. They could even modify the sampler so this sequence of tokens is never selected. On the input side, they could check that the embedding of the input is not within some threshold of meaning of a jailbreak.
- minimaxir 3y ago> ended any sessions where the secret text was about to be leaked As ChatGPT streams live responses, that would create significant latency for the other 99.9% of users. It's not an easy product problem to solve. > On the input side, they could check that the embedding of the input is not within some threshold of meaning of a jailbreak. That is more doable, but people have made creative ways to jailbreak that a simple embedding check won't catch.
- simonw 3y agoOne thing I've learned about prompt injection is that any techniques that seem like they should be obvious and easy very rarely actually work.
- deleted 3y ago[deleted]