11 ms·
Poisoning Well
- tqwhite 1y agoI continue to have contempt for the "I'm not contributing to the enrichment of our newest and most powerful technology" gang. I do not accept the assertion that my AI should have any less access to the internet that we all pay for than I do. If guys like this have their way, AI will remain stupid and limited and we will all be worse off for it.
- thoroughburro 1y agoI do not accept the assertion that the content I host is anything other than my own to serve to whom I wish. Get your hands off my belongings, freeloader. Or, shorter: I hold you in as much contempt as you hold me. > If guys like this have their way, AI will remain stupid and limited AI doesn’t have a right to my stuff just because it’ll suck without it.
- tqwhite 1y agoYou put it on the public internet, the public has the right to see it, take whatever lessons it can and do what it wants. It's the intrinsic nature of the internet. It has never been ok to say, "[this kind of person] is not allowed to look at my page nor, "you can only look at my page with [this web browser or other tool]".
- thoroughburro 1y ago> It has never been ok to say, "[this kind of person] is not allowed to look at my page nor, "you can only look at my page with [this web browser or other tool]". What? Sure it has. “This page best viewed in Internet Explorer.” “JavaScript required.” “Members only.” “Aggressive scrapers blocked.” “Random country blocked just for having higher proportion undesirable traffic.” All bog standard for decades.
- IncreasePosts 1y agoOkay, what percent of content is poison like this? Way less than 1%? If AI is so smart maybe it can figure out what content in a training set is jibberish and what isn't. Anyway, a big problem people have isn't "AI bad", it's "AI crawlers bad", because they eat up a huge chunk of your bandwidth by behaving badly for content you intend to serve to humans.
- tqwhite 1y agoThe amount of bandwidth they use is trivial. That's not an issue. I don't have any idea if the poisoning idea is real or not. I express contempt for those that think it's a good idea. It's not just my support for AI learning from the public internet, there are also people who will see it. It's unethical.
- nkrisc 1y ago> I continue to have contempt for the "I'm not contributing to the enrichment of our newest and most powerful technology" gang. Ok, when do I get paid for my contribution?
- tqwhite 1y agoIf you have posted public web pages then the fact of your being able to be seen by people using the public network infrastructure is your payment.
- tqwhite 1y agoIf you put stuff outside, in public, then it is correct for people to observe and learn from it. You should be happy that your web presence is enriching our culture. If not, you're selfish and I think you're bad. If you don't want it to contribute to the public use, make a login page. I see no reason for you to get the advantage of a public access without contributing to the community.
- hosh 1y agoThere is a project called Iocaine that does something similar while trying to minimize resource use.
- agar 1y agoIsn't the Iocaine developer from Australia?
- wilg 1y agoIn my opinion, colonialism was significantly worse than web crawlers being used to train LLMs.
- amelius 1y agoWe're only at the beginning.
- nvader 1y agoA link to the poisoned version of the same article: https://heydonworks.com/nonsense/poisoning-well/ https://heydonworks.com/nonsense/poisoning-well/
- 1970-01-01 1y agoCrazy how close to coherent it reads, yet it clearly is gibberish.
- heelix 1y agoSimilar thought when I looked at the above link. My pre-coffee brain was parsing along initially, and then it seemed to drift into silly - all while looking correct. I got to believe that sort of input would be toxic to a model.
- rightbyte 1y agoYe there is this flow. If you remember how ChatGPT was at release, it like slided into gibberish and you wouldn't instantly notice but a sentence or two in.
- 8organicbits 1y agoThe robots.txt used here only tells GoogleBot to avoid /nonsense/. It would be nice to tell other web crawlers too otherwise your poisoning everyone but Google, not just crawlers that ignore robots.txt
- johnnienaked 1y ago>Since most of what they consume is on the open web, it’s difficult for authors to withhold consent without also depriving legitimate agents (AKA humans or “meat bags”) of information. It shouldn't be difficult at all. When you record original music, or write something down on paper, it's instantly copyrighted. Why shouldn't that same legal precedent apply to content on the internet? This is half a failure of elected representatives to do their jobs, and half amoral tech companies exploiting legal loopholes. Normal people almost universally agree something needs to be done about it, and the conversation is not a new one either.
- karahime 1y agoPersonally, I would not want to be on the side of people openly saying that they are poisoning the well.
- rideontime 1y agoA well drilled on my property, without even asking for permission, let alone receiving it? I'll happily take that side.
- djoldman 1y ago> I’m not drab I want the base to end this nobody. I don't know why but the examples are hilarious to me.
- joncp 1y agoThat’ll end up in an arms race where you refine the gibberish to be more and more believable while the crawlers get better and better at detecting poison wells. The end state is where your fake pages are so close to the real thing that humans can’t tell the difference
- robocat 1y agoHumans have been refining their gibberish for centuries. I admire your idealism that "the real thing" is coherently different from good gibberish. The poisoned version of the same article is great: https://heydonworks.com/nonsense/poisoning-well/ https://heydonworks.com/nonsense/poisoning-well/ (I love me some surrealism so gibberish is something I sometimes choose to input into my own model in my head). The scary part of AI is that it shows how crappy most of the training material is.
- deleted 1y ago[deleted]
- wolvesechoes 1y agoI am fine with poisoning the Google's well too. I rely on people recommending my blog to other people if they find it interesting, so ranking means nothing to me.
- bboygravity 1y agoI find this whole anti-LLM stance so weird. It kind of feels like trying to build robot distractions into websites to distract search engine indexers in the 2000's or something. Like why? Don't you want people to read your content? Does it really matter that meat bags find out about your message to the world through your own website or through an LLM? Meanwhile, the rest of the world is trying to figure out how to deliberately get their stuff INTO as many LLMs as fast as possible.
- HankStallone 1y ago> Don't you want people to read your content? People, yes. Well-behaved crawlers that follow established methods to prevent overload and obey established restrictions like robots.txt, yes. Bots that ignore all that and hammer my site dozens of times a second, no. I don't see the odds of someone finding my site through an LLM being high enough to put up with all the bad behavior. In my own use of LLMs, they only occasionally include links, and even more rarely do I click on one. The chance that an LLM is going to use my content and include a link to it, and that the answer will leave something out so the person needs to click the link for more, seems very remote.
- timdiggerm 1y agoIf it's ad-supported or I'm seeking donations, I only want people reading it on my website. Why would I want people to access it through an LLM?
- LtWorf 1y agoI don't want having to pay extra money for vibe-coded LLMs companies bots to scrape my website constantly, ignoring cache headers and the likes. Every single person who has wrote a book is happy if others read their book. They might be less enthusiastic about printing million copies and shipping them to random people with their own money.
- deleted 1y ago[deleted]
- wonger_ 1y ago
- simonw 1y agoThere are two common misconceptions in this post. The first isn't worth arguing against: it's the idea that LLM vendors ignore your robots.txt file even when they clearly state that they'll obey it: https://platform.openai.com/docs/bots https://platform.openai.com/docs/bots Since LLM skeptics frequently characterize all LLM vendors as dishonest mustache-twirling cartoon villains there's little point trying to convince them that companies sometimes actually do what they say they are doing. The bigger misconception though is the idea that LLM training involves indiscriminately hoovering up every inch of text that the lab can get hold of, quality be damned. As far as I can tell that hasn't been true since the GPT-3 era. Building a great LLM is entirely about building a high quality training set. That's the whole game! Filtering out garbage articles full of spelling mistakes is one of many steps a vendor will take in curating that training data.
- charles_f 1y agoI somewhat agree with your viewpoint on copyright, but what terrifies me is VCs like a16z or Sequoia simultaneously backing up large LLMs profiting from ignoring copyright and media firms where they'll use whatever power and lobby they have to protect copyright. I don't think the content I produce is worth that much, I'm glad if it can serve anyone, but I find amusing the idea to poison the well
- vintermann 1y agoThere are definitively scrapers that ignore your robots.txt file. Whether they're some "Enemy State" LLM outfit, an "Allied State" corporation outsourcing their dirty work a step or two, or just some data hoarder worried that the web as we know it is going away soon, everyone is saying they're a problem lately, I don't think everyone is lying. But it's certainly also true that anyone feeding the scrapings to an LLM will filter it first. It's very naive of this author to think that his adlib-spun prose won't get detected and filtered out long before it's used for training. Even the pre-LLM internet had endless pages of this sort of thing, from aspiring SEO spammers. Yes, you're wasting a bit of the scraper's resources, but you can bet they're already calculating in that waste.
- 1y ago
- deadbabe 1y agoNot every bot that ignores your robots.txt is necessarily using that data. What some bots do is they first scrape the whole site, then look at which parts are covered by robots.txt, and then store that portion of the website under an “ignored” flag. This way, if your robots.txt changes later, they don’t have to scrape the whole site again, they can just turn off the ignored flag.
- imtringued 1y agoYour post-rationalization just doubles down on the stance that these crawlers are abusive and poorly developed. You're also under the blatantly wrong misconception that people are worried about their data, when they are worried about the load of a poorly configured crawler. The crawler will scrape the whole website on a regular interval anyway, so what is the point of this "optimization" that optimizes for highly infrequent events?
- nvader 1y agoNot every intruder who enters your home is necessarily a burglar.
- cyphar 1y agoAh, so the NSA defence then -- "it's not bulk collection because it only counts as collection when we look at it".
- neuroelectron 1y agoKind of too late for this. The ground truth of models has already been established. That' why we see models converging. They will automatically reject this kind of poison.
- sevensor 1y agoI don’t know about that. Have you seen their output? They’re poisoning their own well with specious nonsense text.
- nine_k 1y agoThis will remain so as long as the models don't need to ingest any new information. If most novel texts will appear with slightly more insidious nonsense mirrors, LLMs would either have to stay without this knowledge, or start respecting "nofollow".
- blagie 1y agoIt's competition. Poison increases in toxicity over time. I could generate subtly wrong information on the internet LLMs would continue to swallow up.
- wewtyflakes 1y agoYes, but so would people, so what's the point of this unless you just dislike everybody (and if so, that's fair too I suppose)?
- blagie 1y agoThe whole design was based on things to make pages not discoverable except to sketchy AI companies who violate web norms.
- latexr 1y ago> I could generate subtly wrong information on the internet There’s already a website for that. It’s called Reddit.
- fastball 1y ago> According to Google, it’s possible to verify Googlebot by matching the crawler’s IP against a list of published Googlebot IPs. This is rather technical and highly intensive Wat. Blocklisting IPs is not very technical (for someone running a website that knows + cares about crawling) and is definitely not intensive. Fetch IP list, add to blocklist. Repeat daily with cronjob. Would take an LLM (heh) 10 seconds to write you the necessary script.
- simonw 1y agoLooks like OpenAI publish the IPs of their training data crawler too: https://openai.com/gptbot.json https://openai.com/gptbot.json
- kinix 1y agoFrom how I read it, the author seems to be suggesting that this list of IPs be on an allowlist as they see Google as "less nefarious". As such, sure, allowing google IPs is as easy as allowing all IPs, but discerning who are "nefarious actors" is probably harder. A more tounge-in-cheek point: all scripts take an LLM ~10 seconds to write, doesn't mean it's right though.
- kitku 1y agoThis reminds me of the Nepenthes tarpit [1], which is an endless source of ad-hoc generated garbled mess which links to itself over and over. Probably more effective at poisoning the dataset if one has the resources to run it. [1]: https://zadzmo.org/code/nepenthes/ https://zadzmo.org/code/nepenthes/
- 8organicbits 1y agoDo we know if LLM scrapers are running JavaScript on the pages? If they are, maybe it's worth offloading the Markov model to the client side.
- fleebee 1y agoI'm running Iocaine[1] which is essentially the same thing on my tiny $3/mo VPS and it's handling crawlers bombarding the honeypot with ~12 requests per second just fine. It's using about 30 MB of RAM. [1]: https://iocaine.madhouse-project.org/ https://iocaine.madhouse-project.org/
- treetalker 1y agoOdorless, tasteless, and among the more deadly poisons known to crawlers!
- BrenBarn 1y agoUnfortunately they will spend the next several years building up an immunity.
- protocolture 1y ago>One of the many pressing issues with Large Language Models (LLMs) is they are trained on content that isn’t theirs to consume. One of the many pressing issues is that people believe that ownership of content should be absolute, that hammer makers should be able to dictate what is made with hammers they sell. This is absolutely poison as a concept. Content belongs to everyone. Creators of content have a limited term, limited right to exploit that content. They should be protected from perfect reconstruction and sale of that content, and nothing else. Every IP law counter to that is toxic to culture and society.
- computerthings 1y ago[dead]
- kstrauser 1y agoTencent scrapers are hitting my little Forgejo site 4 times a second, 24/7. I pay for that bandwidth. Platitudes sound great, but this isn’t a lofty “drinking from the public well”. This is bastard operators taking a drink and pooping in it. My thoughts will have more room for nuance when they stop abusing the hell out of my resources they’re “borrowing”.
- protocolture 1y agoBeing a dick while scraping isnt really the same question as the use of that data. Anyway the answer is block em.
- psychoslave 1y agoWhy are there even doing so? This doesn’t feel like something that can even bring any value downstream to their own selfish pipelines, or am I missing something?
- kstrauser 1y agoNo! They’re constantly hitting the same stupid URL (“show me this file in this commit in this repo with these 47 query params”) from a few thousand IPs in China and Brazil, with user agents showing an iPod or a Linux desktop running Opera 3. I wrote a little script where I throw in an IP and it generates a Caddy IP-matcher block with an “abort” rule for every netblock in that IP’s ASN. I’m sure there are more elegant ways to share my work with the world while blocking the scoundrels, but this is kind of satisfying for the moment.
- Popeyes 1y agoWhat is this war about? I was looking at another thread about how Wikipedia was the best thing on the internet. But they only got the head start by taking copy of Encyclopedia Britannica and everything else is a And now the corpus is collected, what difference does a blog post make, does it nudge the dial to comprehension 0.001% in a better direction? How many blog posts over how many weeks makes the difference.
- collinmcnulty 1y agoIt is about imposing costs on poorly behaved scraping in an attempt to change the scrapers behavior, under the assumption that the scrapers' creators are anti-social but economically rational. One blog doesn't make a huge difference but if enough new blogs contain tarpits that cost the scraper as much as the benefit of 100 other non-tarpit blogs, maybe the calculus for doing any new scraping changes and the scrapers start behaving.
- nvader 1y ago> they only got the head start by taking copy of Encyclopedia Britannica Wikipedia used a version of Encyclopedia Britannica that was in the public domain. Go thou and do likewise.
- simonw 1y agoThis is the first I've heard of Wikipedia starting with a copy of Britannia. Where did you see that?
- simonw 1y agoOK, found it: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Encyclopaedia_Britannica https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Encyclop... "Starting in 2006, much of the still-useful text in the 1911 Encyclopaedia was adapted and absorbed into Wikipedia. Special focus was given to topics that had no equivalent in Wikipedia at the time. The process we used is outlined at the end of this page." Wikipedia started in 2001. Looks like they absorbed a bunch of out-of-copyright Britannica 1911 content five years later. There are still 13,000 pages on Wikipedia today that are tagged as deriving from that project: https://en.m.wikipedia.org/wiki/Template:EB1911 https://en.m.wikipedia.org/wiki/Template:EB1911