6 ms·
The point is that some use toxicity as a deliberate weapon, and that weapon can now be encoded into the LLM via training by those same aggressors. This multipli
by afterburner 3y ago
The point is that some use toxicity as a deliberate weapon, and that weapon can now be encoded into the LLM via training by those same aggressors. This multiplies their reach with minimal effort.
- sillysaurusx 3y agoI’m skeptical anyone has ever done this successfully. Once there are examples people can point to, this talking point might have merit. But this fear dates back to at least 2019, and as far as I can tell it’s still unfounded.
- ajdecon 3y agoHow about Microsoft Tay? https://en.m.wikipedia.org/wiki/Tay_(chatbot) https://en.m.wikipedia.org/wiki/Tay_(chatbot) While the underlying model is certainly different, and my understanding is that current LLMs don’t learn “live”, the principle seems worth keeping in mind.
- jabradoodle 3y agoYou are sceptical that people will use LLMs for nefarious purposes?
- s1artibartfast 3y agothe LLM were already trained on data with toxicity, This is why they have the weightings to know how to respond. They aren't really teaching the model anything it hasnt seen before. If you want toxic output, you can get it.
- courseofaction 3y agoFor little effort the defending dev can ask a second layer of LLM to check if an output is explicitly toxic, filter it, and nullify the red team. Filtering shitty content is easier than creating it with a properly constructed LLM system, the complaints about toxic outputs seem to me to be analogous to an electrical engineer complaining that the voltage from the mains is wrong for their device, but refusing to google what an (electrical) transformer is. Toxic writing pre-exists LLMs. LLMs output writing. This is not a new problem, but we have a new solution - LLM filtering.
- Timon3 3y ago> Filtering shitty content is easier than creating it with a properly constructed LLM system Can you explain this? It feels completely wrong - even OpenAI, who probably have invested the most, can't filter out all "shitty content". An LLM can on the other hand create "shitty content" incredibly easy - even if the creators try to stop it! So how is filtering easier than creating?
- psychphysic 3y agoLLMs don't even register the list of deliberate weapons. Just seems like pearl clutching at it's finest.