3 ms·
This is what the Waluigi effect is, since it isn't described at the top: > The Waluigi Effect: After you train an LLM to satisfy a desirable property P, then i
by Jensson 4y ago
This is what the Waluigi effect is, since it isn't described at the top:
> The Waluigi Effect: After you train an LLM to satisfy a desirable property P, then it's easier to elicit the chatbot into satisfying the exact opposite of property P.
Basically, the chatbot will often do the opposite of what you say.
- bentcorner 4y agoIMO it's more like if you tell the LLM to never talk about Pink Elephants, it will become easier to get it to talk about Pink Elephants later. (It is easier to get an anti-Pink Elephant model to talk about Pink Elephants than it is to get a neutral model to talk about Pink Elephants)
- colechristensen 4y agoRight, in order not to talk about pink elephants you have to be particularly interested in pink elephants and therefore you gather a lot of pink elephant knowledge. I have similar thoughts about swearing being kept alive by teaching children not to say "bad" words and various kinds of bigotry being amplified at this point by people trying to fight against it.
- lalos 4y agoReads like the Streisand effect
- tmalsburg2 4y agoThis may happen simply because LLM, like humans, are bad a negation.