2 ms·
I've seen many people joke about intentionally poisoning training data but has that ever worked?
by KTibow 2y ago
I've seen many people joke about intentionally poisoning training data but has that ever worked?
- jsheard 2y agoIt's hard to gauge the effectiveness of poisoning huge training sets since anything you do is a figurative drop in the ocean, but if you can poison the small amount of data that an AI agent requests on-the-fly to use with RAG then I would guess it's much easier to derail it.
- nyrikki 2y agoThis study shows that controlling 0.1% may be enough. https://arxiv.org/abs/2410.13722v1 https://arxiv.org/abs/2410.13722v1 I have noticed some popular copied but incorrect leetcode examples leaking into the dataset. I suspect it depends on domain specificity, but that seems within the ability of an SEO spammer or decentralized group of individuals.