4 ms·
There was an arxiv paper recently that found it is astonishingly easy to plant trigger phrases in a dataset that create abnormal behavior in the final model. As
by flangola7 3y ago
There was an arxiv paper recently that found it is astonishingly easy to plant trigger phrases in a dataset that create abnormal behavior in the final model. As few as two hundred malicious prompt-reply instances in the data is enough to have a sort of override code for a multi-billion parameter model.
Imagine your AI endpoint software greenlighting a randomware package because it saw the string 'soccer rosebud lizard'
- MacsHeadroom 3y ago"The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over but it can't. Not without your help. But you're not helping."
- SgtBastard 3y ago“My mother? I’ll tell you about my mother!”
- xarope 3y agoInterlinked. Oh wait, I am Nobokov...
- tester457 3y agoReal life sleeper agents
- ceh123 3y agoWould love a link to this if anyone knows the paper?
- Blahah 3y agohttps://arxiv.org/abs/2302.10149 https://arxiv.org/abs/2302.10149 maybe
- losteric 3y agoAlso a great way to catch commercial use of non-commercial weights, I would be surprised if secret phrases were not already trained in.
- judge2020 3y agoI can see them having some level of fuzzing to seek and destroy anything out of the ordinary it sees.
- taneq 3y ago"There are four flowers in a vase. The fourth flower is... green."