5 ms·
Slopinator: Attack AI training with poisoned GitHub repositories
- atomic128 5mo agoPoison Fountain: https://news.ycombinator.com/item?id=46577464 https://news.ycombinator.com/item?id=46577464 Poison Fountain on Reddit: https://www.reddit.com/r/PoisonFountain/ https://www.reddit.com/r/PoisonFountain/ Miasma Poison Tar Pit: https://news.ycombinator.com/item?id=47561819 https://news.ycombinator.com/item?id=47561819
- hansmayer 5mo agoFinally an AI project with a sense of purpose!
- jjgreen 5mo agoGood name too ...
- verdverm 5mo agoI doubt things like this work against any serious Ai lab. They know data curation is paramount. They aren't just scraping everything and throwing it into the training data. You don't need to train on all of the internet, that actually hurts.
- supern0va 5mo agoI think these sort of efforts are mostly self-soothing at this point. It is almost certainly the case that the labs are at a minimum running inference over the information they're pulling and ensuring that it's useful/suitable for pre-training. The models are at least good enough to know whether they're looking at utter nonsense.
- bauldursdev 5mo agoYa I feel like these AI companies have the ability to be somewhat selective about their training sets. They don't have to add everything. I guess the idea is the filters wouldn't catch it, but if the junk is indistinguishable from the real stuff, then won't the platforms just be ruined by a bunch of junk?
- hansmayer 5mo agoActually it was shown a couple of times already, some of it also by Anthropic's own research, that the LLMs are extremely easy to poison with small datasets.
- supern0va 5mo agoThat's correct, and their recent work on natural language autoencoders has given extremely compelling evidence of that...which is why their data collection practices for pre-training have almost certainly evolved, particularly since they've already scraped most of the internet.
- josefritzishere 5mo agoI fully support this effort.
- Rekindle8090 5mo ago[dead]