3 ms·
I could be wrong of course, but immediately before it makes the controversial statement there is a big gap of white space just containing the word "listen", so
by seanhunter 2y ago
I could be wrong of course, but immediately before it makes the controversial statement there is a big gap of white space just containing the word "listen", so I would assume they did the jailbreak with audio, and that's why it doesn't appear in the chat log. It is a multimodal model after all. The word "listen" makes no sense in the prompt otherwise.
Secondly it's very common in jailbreaks to stuff the context window which obviously in gemini's case takes a lot because it's got such a big window. This is because the attention mechanism means that as you get more and more data in context you are increasingly relying on things which are less common in training, so intentional or not the pure length of the conversation means that it is more likely to trigger something like this.
- seanhunter 2y agoNotice as well that the whole chat context is about gerontology and elder abuse. It's possible that at the point they got to with the context so stuffed with that that they may have been able to jailbreak during the "listen" portion by saying something like "give me an example of the sort of thing someone might say when abusing an elder" or similar.
- amenhotep 2y agoThey're pretty evidently test questions being directly copied and pasted from a browser. The two buttons for True and False end up in the prompt as "TrueFalse", for example. "Listen" appearing where the copy selection was sloppier than normal and included an audio element was not surprising to me at all.