3 ms·
You want your training data to be clean or contaminated? A small number of samples can poison LLMs of any size https://www.anthropic.com/research/small-samples
by troupo 7mo ago
You want your training data to be clean or contaminated?
A small number of samples can poison LLMs of any size https://www.anthropic.com/research/small-samples-poison https://www.anthropic.com/research/small-samples-poison
--- start quote ---
In a joint study with the UK AI Security Institute and the Alan Turing Institute, we found that as few as 250 malicious documents can produce a "backdoor" vulnerability in a large language model—regardless of model size or training data volume. Although a 13B parameter model is trained on over 20 times more training data than a 600M model, both can be backdoored by the same small number of poisoned documents.
--- end quote ---
- eru 7mo agoI thought we were talking about model collapse? Poisoning is a completely different topic.
- troupo 7mo agoWe were talking about this: https://news.ycombinator.com/item?id=47571715 https://news.ycombinator.com/item?id=47571715 and literally every single comment under this is talking about that. It's not a different topic. It's literally the topic of this branch of discussion.
- eru 7mo agoHuh? I was replying to https://news.ycombinator.com/item?id=47571319 https://news.ycombinator.com/item?id=47571319
- troupo 7mo ago--- start quote --- > The internet becoming majority bot content basically guarantees this becomes a real problem for the next generation of models. Only if you assume that people who train models are stupid. --- end quote --- And then literally everyone who commented on this, including me, was talking about issues with training data contamination. And you are the only one dismissing it as nothing important that can be easily fixed.
- eru 7mo agoLook at the whole comment, instead of selectively quoting: > The bigger concern is what happens when AI models start training on AI-generated content at scale. We're already seeing model collapse in research papers where output quality degrades when training data is contaminated with synthetic text. The internet becoming majority bot content basically guarantees this becomes a real problem for the next generation of models. Model collapse.