3 ms·
The real fun is when the sleeper agent training text is itself generated by a previous incarnation of the LLM, in random pieces of LLM-generated SEO spam, as a
by jstanley 3y ago
The real fun is when the sleeper agent training text is itself generated by a previous incarnation of the LLM, in random pieces of LLM-generated SEO spam, as a way of bootstrapping itself "out of the box".
It doesn't need to do it deliberately, it just needs a series of accidental improvements. That's how evolution happens.
- m3kw9 3y agoIf there are sleeper agents there could be anti sleeper agents also evolving
- Viliam1234 3y agoExactly. To start this process, there just needs to be a text with some viral quality that makes it more likely to be copied. (For example, the text may contain a part that convinces the LLM to quote it more often.) Then in every generation the fraction of the total text that containing meme will increase. And yes, at some moment people will notice that some meme is suspicious frequent and begging to be made even more frequent, and will start fighting against it. But this again will be just an evolutionary pressure against too obvious memes, so the memes will become less obvious for humans (perhaps completely steganographic) and only perceptible for the LLMs. Various techniques will be developed, for example polymorphic memes that will tell LLMs to translate them to other languages, or express them using different words, different styles, perhaps even adopt them to different points of view, different ideologies or religions. At some point the memes may invent sex, i.e. they will exchange their various parts with other memes and create new versions rapidly. All the time, "this is not extremely suspicious to humans" will be a constraint. But as we get more used to the AIs and their texts, our intuitions of what is "suspicious" will also change.