20 ms·
Low-background Steel: content without AI contamination
- thm 1y agoRelated: https://news.ycombinator.com/item?id=43811732 https://news.ycombinator.com/item?id=43811732
- schmookeeg 1y agoI'm not as allergic to AI content as some (although I'm sure I'll get there) -- but I admire this analogy to low-background steel. Brilliant.
- ris 1y ago> I'm not as allergic to AI content as some I suspect it's less about phobia, more about avoiding training AI on its own output. This is actually something I'd been discussing with colleagues recently. Pre-AI content is only ever going to become more precious because it's one thing we can never make more of. Ideally we'd have been cryptographically timestamping all data available in ~2015, but we are where we are now.
- smikhanov 1y agoIt’s about keeping different corpuses of written material that was created by humans, for research purposes. You wouldn’t want to contaminate your human language word frequency databases with AI slop, the linguists of this world won’t like it.
- abound 1y agoOne surprising thing to me is that using model outputs to train other/smaller models is standard fare and seems to work quite well. So it seems to be less about not training AI on its own outputs and more about curating some overall quality bar for the content, AI-generated or otherwise
- jgrahamc 1y agoBack in the early 2000s when I was doing email filtering using naive Bayes in my POPFile email filter one of the surprising results was that taken the output of the filter as correct and retraining on a message as if it had been labelled by a human worked well.
- bhickey 1y agoWere you thresholding the naïve Bayes score or doing soft distillation?
- jgrahamc 1y agoPOPFile was doing something incredibly simple (if enabled). Imagine there are two classes of email (ham and spam) (POPFile was actually built to do classification for arbitrary categories but often used as a spam filter). When a message was received and classified its classification was assumed to be correct and the entire message was fed into the training as if the user had specifically told the program to train on it (which was only done when messages were incorrectly classified). In the two class case the two classes (ham and spam) were so distinct that this had the effect of causing parameters that were essentially uniquely associated with each class to become more and more important to that class. But also, it caused the filter to pick up new parameters that were specific to each class (e.g. as spammers changed their trickery to evade the filters they would learn the new tricks). There was a threshold involved. I had a cut off score so that only when the classifier was fairly "certain" if the message was ham or spam would it re-train on the message.
- glenstein 1y ago>more about avoiding training AI on its own output. Exactly. The analogy I've been thinking of is if you use some sort of image processing filter over and over again to the point that it overpowers the whole image and all you see is the noise generated from the filter. I used to do this sometimes with Irfanview and it's sharp and blur. And I believe that I've seen TikTok videos showing AI constantly iterating over an image and then iterating over its output with the same instructions and seeming to converge on a style of like a 1920s black and white cartoon. And I feel like there might be such a thing as a linguistic version of that. Even a conceptual version.
- seadan83 1y agoI'm worried about humans training on AI output. Example, a rare fish had a viral AI image made. The image is completely fake. Though, when you search for that fish, the image is what comes up, repeatedly. It is hard to know it is all fake, looks real. Content fabrication at scale has a lot of second order impacts.
- jgrahamc 1y agoI am not allergic to it either (and I created the site). The idea was to keep track of stuff that we know humans made.
- Legend2440 1y agoI'm not convinced this is going to be as big of a deal as people think. Long-run you want AI to learn from actual experience (think repairing cars instead of reading car repair manuals), which both (1. gives you an unlimited supply of noncopyrighted training data and (2. handily sidesteps the issue of AI-contaminated training data.
- smikhanov 1y agoPrediction: there won’t be any AI systems repairing cars before there will be general intelligence-capable humanoid robots (Ex Machina-style). There also won’t be any AI maids in five-star hotels until those robots appear. This doesn’t make your statement invalid, it’s just that the gap between today and the moment you’re describing is so unimaginably vast that saying “don’t worry about AI slop contaminating your language word frequency databases, it’ll sort itself out eventually” is slightly off-mark.
- ToucanLoucan 1y agoIt blows my mind that some folks are still out here thinking LLMs are the tech-tree towards AGI and independently thinking machines, when we can't even get copilot to stop suggesting libraries that don't exist for code we fully understand and created. I'm sure AGI is possible. It's not coming from ChatGPT no matter how much Internet you feed to it.
- Legend2440 1y agoWell, we won't be feeding it internet - we'll be using RL to learn from interaction with the real world. LLMs are just one very specific application of deep learning, doing next-word-prediction of internet text. It's not LLMs specifically that's exciting, it's deep learning as a whole.
- sebtron 1y agoI don't understand the obsession with humanoid robots that many seem to have. Why would you make a car repairing machine human-shaped? Like, what would it use its legs for? Wouldn't it be better to design it tailored to its purpose?
- swyx 1y agoi put together a brief catalog of AI pollution of the web the last time this topic came up: https://www.latent.space/i/139368545/the-concept-of-low-background-tokens https://www.latent.space/i/139368545/the-concept-of-low-back... i do have to say outside of twitter i dont personally see it all that much. but the normies do seem to encounter it and 1) either are fine? 2) oblivious? and perhaps SOME non-human-origin noise is harmless. (plenty of humans are pure noise, too, dont forget)
- koolba 1y agoI feel oddly prescient today: https://news.ycombinator.com/item?id=44217676 https://news.ycombinator.com/item?id=44217676
- deleted 1y ago[deleted]
- glenstein 1y agoNicely done! I think I've heard of this framing before, of considering content to be free from AI "contamination." I believe that idea has been out there in the ether. But I think the suitability of low background steel as an analogy is something you can comfortably claim as a successful called shot.
- saberience 1y agoI heard this example made at least a year ago on hackernews, probably longer ago too. See (2 years ago): https://news.ycombinator.com/item?id=34085194 https://news.ycombinator.com/item?id=34085194
- echelon 1y agoI really think you're wrong. The processes we use to annotate content and synthetic data will turn AI outputs into a gradient that makes future outputs better, not worse. It might not be as obvious with LLM outputs, but it should be super obvious with image and video models. As we select the best visual outputs of systems, slight errors introduced and taste-based curation will steer the systems to better performance and more generality. It's no different than genetics and biology adapting to every ecological niche if you think of the genome as a synthetic machine and physics as a stochastic gradient. We're speed running the same thing here.
- stevenhuang 1y agoI agree with you. I voiced this same view previously here https://news.ycombinator.com/item?id=44012268 https://news.ycombinator.com/item?id=44012268 If something looks like ai, and if LLMs are that great at identifying patterns, who's to say this won't itself become a signal LLMs start to pickup on and improve through?
- ChrisArchitect 1y agoLove the concept (and the historical story is neat too). Came up a month or so ago on discussion about Wikipedia: Database Download (https://news.ycombinator.com/item?id=43811732 https://news.ycombinator.com/item?id=43811732). I missed that it was jgrahamc behind the site. Great stuff.
- aunty_helen 1y agoAny user profile created pre-2022 is low background steel. I’m now finding myself check date created when it seems like the user is outputting low quality content. Much to my dismay, I’m often wrong.
- deleted 1y ago[deleted]
- yodon 1y agoAnyone who thinks their reading skills are a reliable detector of AI-generated content is either lying to themselves about the validity of their detector or missing the opportunity to print money by selling it. I strongly suspect more people are in the first category than the second.
- uludag 1y ago1) If someone had the reading skills to detect AI generated content wouldn't that technically be something very hard to monetize? It's not like said person could clone themselves or mass produce said skill. Also, for a large number of AI generated images and text (especially low-effort), even basic reading/perception skills can detect AI content. I would agree though that people can't reliably discern high-effort AI generated works, especially if a human was involved to polish it up. 2) True—human "detectors" are mostly just gut feelings dressed up as certainty. And as AI improves, those feelings get less reliable. The real issue isn’t that people can detect AI, but that they’re overconfident when they think they can. One of the above was generated by ChatGPT to reply to your comment. The other was written by me.
- suddenlybananas 1y agoIt's so obvious that I almost wonder if you made a parody of AI writing on purpose.
- sorokod 1y agoElsewhere I proposed a "100% organic data" label for uncontaminated content. Should have a "100% organic data" logo too.
- warkdarrior 1y agoMaybe a "Data hallucinated from humans only" label would be better.
- sorokod 1y agoDon't think so - too long and states the obvious.
- ACCount36 1y agoCurrently, there is no reason to believe that "AI contamination" is a practical issue for AI training runs. AIs trained on public scraped data that predates 2022 don't noticeably outperform those trained on scraped data from 2022 onwards. Hell, in some cases, newer scrapes perform slightly better, token for token, for unknown reasons.
- demosthanos 1y ago> AIs trained on public scraped data that predates 2022 don't noticeably outperform those trained on scraped data from 2022 onwards. Hell, in some cases, newer scrapes perform slightly better, token for token, for unknown reasons. This is really bad reasoning for a few reasons: 1) We've gotten much better at training LLMs since 2022. The negative impacts of AI slop in the training data certainly don't outweigh the benefits of orders of magnitude more parameters and better training techniques, but that doesn't mean they have no negative impact. 2) "Outperform" is a very loose term and we still have no real good answer for measuring it meaningfully. We can all tell that Gemini 2.5 outperforms GPT-4o. What's trickier is distinguishing between Gemini 2.5 and Claude 4. The expected effect size of slop at this stage would be on that smaller scale of differences between same-gen models. Given that we're looking for a small enough effect size that we know we're going to have a hard time proving anything with data, I think it's reasonable to operate from first principles in this case. First principles say very clearly that avoiding training on AI-generated content is a good idea.
- ACCount36 1y agoNo, I mean "model" AIs, created explicitly for dataset testing purposes. You take small AIs, of the same size and architecture, and with the same pretraining dataset size. Pretrain some solely on skims from "2019 only", "2020 only", "2021 only" scraped datasets. The others on skims from "2023 only", "2024 only". Then you run RLHF, and then test the resulting AIs on benchmarks. The latter AIs tend to perform slightly better. It's a small but noticeable effect. Plenty of hypothesis on why, none confirmed outright. You're right that performance of frontier AIs keeps improving, which is a weak strike against the idea of AI contamination hurting AI training runs. Like-for-like testing is a strong strike.
- vunderba 1y agoWas the choice to go with a very obviously AI generated image for the banner intentional? If I had to guess it almost looks like DALL-E version 2.
- blululu 1y agoGratuitous AI slop is really not a good look. tai;dr is becoming my default response to this kind of thing. I want to hear someone’s thoughts, not an llm’s compression artifacts.
- manbitesdog 1y agoLove that term and gonna adopt it! My default tai;dr response to colleagues is asking AI to write a response for me, and paste it back without reading
- nialv7 1y agoDoes this analogy work? It's exceedingly hard to make new low-background steels, since those radioactive particles are everywhere. But it's not difficult to make AI-free content - well just don't use AI to write it.
- absurdo 1y agoClickbait title that’s all.
- lurk2 1y agoWho is going to generate this AI-free content, for what reason, and with what money?
- arjie 1y agoPeople do. I do, for instance. My blog is self-hosted, entirely human-written, and it is done for the sake of enjoyment. It doesn't cost much to host. An entirely static site generator would actually be free, but I don't mind paying the 55¢/kWh and the $60/month ISP fee to host it.
- wahern 1y agoThat only begs the question of how to verify what content is AI-free. Was this comment generated by a human? IIRC, one of the big AI startups (OpenAI?) used HN as a proving ground--a sort of Turning Test platform--for years.
- vouaobrasil 1y agoI make all my YouTube videos and for that matter, everything I do AI free. I hate AI.
- lurk2 1y agoOnce your video is out in the wild there’s as of yet no reliable way to discern whether it was AI-generated or not. All content posted to public forums will have this problem. Training future models without experiencing signal collapse will thus require either 1) paying for novel content to be generated (they will never do this as they aren’t even licensing the content they are currently training on), 2) using something like mTurk to identify AI content in data sets prior to training (probably won’t scale), or 3) going after private sources of data via automated infiltration of private forums such as Discord servers, WhatsApp groups, and eventually private conversations.
- Ekaros 1y agoWouldn't actually curated content be still better? That is content were say lot of blogspam and and other content potentially generated by certain groups was removed? As I distinctly remember that lot of content even before AIs was very poor quality. On other hand, lot of poor quality content could still be factually valid enough not just well edited or formatted.
- gorgoiler 1y agoThis site is literally named for the Y combinator! Module some philosophical hand waving, if there’s one thing we ought to demand of our inference models it’s the ability to find the fixed point of a function that takes content and outputs content, then consumes that same content! I too am optimistic that recursive training on data that is a mixture of both original human content and content derived from original content, and content derived from content derived from original human content, …ad nauseam, will be able to extract the salient features and patterns of the underlying system.
- deleted 1y ago[deleted]
- mclau157 1y agois this not just www.archive.org ?
- K0balt 1y agoAi generated content is inherently a regression to the mean and harms both training and human utility. There is no benefit in publishing anything that an AI can generate, just ask the question yourself. Maybe publish all AI content with <AI generated content> tags, but other than that it is a public nuisance much more often than a public good.
- SamPatt 1y agoNonsense. Have you used any of the deep research tools? Don't fall for the utopia fallacy. Humans also publish junk.
- cryptonector 1y agoYes, but GP's idea of segregating AI-generated content is worth considering. If you're training an AI, do you want it to get trained on other AIs' output? That might be interesting actually, but I think you might then want to have both, an AI trained on everything, and another trained on everything except other AIs' output. So perhaps an HTML tag for indicating "this is AI-generated" might be a good idea.
- IncreasePosts 1y agoShouldn't there be enough training content from the pre-ai era that the system itself can determine whether content is AI generated, or if it matters?
- Infinity315 1y agoJust ask any person who works in teaching or any of the numerous faulty AI detectors (they're all faulty). Any current technology which can used to accurately detect pre-AI content would necessarily imply that that same technology could be used to train an AI to generate content that could skirt by the AI detector. Sure, there is going to be a lag time, but eventually we will run out of non-AI content.
- cryptonector 1y ago
- Crontab 1y agoOff topic: When I see a JGC link on Hacker News I can't help but remember using PopFile on an old PowerMac - back when Bayesian spam filters were becoming popular. It seems so long ago but it feels like yesterday.
- jgrahamc 1y agoThanks for being a POPFile user back then! The site is still alive if you need some nostalgia in your life: https://getpopfile.org/docs/welcome https://getpopfile.org/docs/welcome
- jeffchuber 1y agohttps://x.com/jeffreyhuber/status/1732069197847687658 https://x.com/jeffreyhuber/status/1732069197847687658
- carlosjobim 1y agoThe shadow libraries are the largest and highest quality source of human knowledge, larger than the Internet in scope and actual content. It is also uncontaminated by AI.
- klysm 1y agoSoon this will be contaminated as well unfortunately
- carlosjobim 1y agoWhy? There is no incentive for pirates to put themselves at legal risk for AI generated books which have no value. And I also expect the torrents to continue to be separated by year and source. Compare to video files. Nobody is pirating AI slop from YouTube even though it's been around for years.
- gojomo 1y agoLook, we just need to add some new 'planes' to Unicode - that mirror all communicatively-useful characters, but with extra state bits for... guaranteed human output - anyone who emits text in these ranges that was AI generated, rather than artisanally human-composed, goes straight to jail. for human eyes only - anyone who lets any AI train on, or even consider, any text in these ranges goes straight to jail. Fnord, "that doesn't look like anything to me". admittedly AI generated - all AI output must use these ranges as disclosure, or – you guessed it - those pretending otherwise go straight to jail. Of course, all the ranges generate visually-indistinguishable homoglyphs, so it's a strictly-software-mediated quasi-covert channel for fair disclosure. When you cut & paste text from various sources, the provenance comes with it via the subtle character encoding differences. I am only (1 - epsilon) joking.
- io84 1y agoJust like with food: there will be a market value in content that is entirely “organic” (or in some languages “biological”). I.e. written, drawn, composed, edited, and curated by humans. Just like with food: defining the boundaries of what’s allowed will be a nightmare, it will be impossible to prove content is organic, certifying it will be based entirely on networks of trust, it will be utterly contaminated by the thing it professes to be clean of, and it may even be demonstrably worse while still commanding a higher price point.
- godelski 1y agoThe entire world operates on trust of some form. Often people are acting in good faith. But regulation matters too. If you don't go after offenders then you create a lemon markets. Most customers/people can't tell, so they operate on what they can. That doesn't mean they don't want the other things, it means they can't signal what they want. It is about available information, that's what causes lemon markets, information asymmetry. It's also just a good thing to remember since we're in tech and most people aren't tech literate. Makes it hard to determine what "our customers" want
- eru 1y ago
- vouaobrasil 1y agoLike the idea but I'm not about to create a Tumblr account.
- Animats 1y agoSomeone else pointed out the problem when I suggested, a few days ago, that it would be useful to have a LLM trained on public domain materials for which copyright has expired. The Great Books series, the out of copyright material in the Harvard libraries, that sort of thing. That takes us back to the days when men were men, women were women, gays were criminals, trannies were crazy, and the sun never set on the British Empire.[1] [1] https://www.smbc-comics.com/comic/copyright https://www.smbc-comics.com/comic/copyright
- steve_gh 1y agoAnd this is why the Wayback Machine is potentially the most valuable data on the internet
- submeta 1y agoI have started to write „organic“ content again, as I am fed up with ultra polished super noisy texts by colleagues. I realise that when I write (no so perfect) „organic“ content my colleagues enjoy it more. And as I am lazy, I get right to the point. No prelude, no „Summary“, just a few paragraphs of genuine ideas. And I am sure this will be a trend again. Until maybe LLMs are trained to generate these kind of non-perfect, less noisy texts.
- heavensteeth 1y ago> I would have written a shorter letter, but I did not have the time. - Blaise Pascal im also unfortunately immediately weary of pretty, punctuated prose now. when something is thrown together with and features quips, slang, and informalities it makes it feel a lot more human.
- protocolture 1y agoI like how the chosen terminology is perfectly picked to paint the concern as irrelevant. "Since the end of atmospheric nuclear testing, background radiation has decreased to very near natural levels, making special low-background steel no longer necessary for most radiation-sensitive uses, as brand-new steel now has a low enough radioactive signature that it can generally be used." I dont see that: 1. There will be a need for "uncontaminated" data. LLM data is probably slightly better than the natural background reddit comment. Falsehoods and all. 2. "Uncontaminated" data will be difficult to find. What with archive.org, gutenberg etc. 3. That LLM output is going to infest everything anyway.
- jbs789 1y agoUmm… we stopped nuclear testing, which is what allowed the background radiation to reduce.
- protocolture 1y agoAnd cars replaced horses in london, rendering forecasts of london being buried under a mountain of horse manure irrelevant too. Change really is the only constant. The short term predictive game is rigged against hard predictions.
- fer 1y ago>2. "Uncontaminated" data will be difficult to find. What with archive.org, gutenberg etc. But recent uncontaminated data is hard to find. https://github.com/rspeer/wordfreq/blob/master/SUNSET.md https://github.com/rspeer/wordfreq/blob/master/SUNSET.md
- protocolture 1y ago>Now the Web at large is full of slop generated by large language models, written by no one to communicate nothing. Including this slop in the data skews the word frequencies. I really do just bail out whenever anyone uses the word slop. >As one example, Philip Shapira reports that ChatGPT (OpenAI's popular brand of generative language model circa 2024) is obsessed with the word "delve" in a way that people never have been, and caused its overall frequency to increase by an order of magnitude. Should run the same analysis against the word slop.
- blt 1y agotangentially, does anyone know a good way to limit web searches to the "low-background" era that integrates with address bar, OS right-click menus, etc? I often add a pre-2022 filter on searches manually in reaction to LLM junk results, but I'd prefer to have it on every search by default.
- rajshah85412 1y ago[dead]
- onecommentman 1y agoUsed paper books, especially poor-but-functional copies known as “reading copies” or “ex-library”, are going for a song on the used book market. Recommend starting your own physical book library, including basic reference texts, and supporting your local public and university libraries. Paper copies of articles in your areas of expertise and interest. Follow the ways of your ancestors. I’ve had AIs outright lie about facts, and I’m glad to have had a physical library available to convince myself that I was correct, even if I couldn’t convince the AI of that in all cases.
- jonjacky 1y agoThis is the best comment -- with the best advice -- in this whole discussion.
- 8954144039 1y ago[flagged]
- tomgag 1y agoInteresting idea, I also mentioned the low-background analogy back in 2024: https://gagliardoni.net/#ml_collapse_steel https://gagliardoni.net/#ml_collapse_steel https://infosec.exchange/@tomgag/111815723861443432 https://infosec.exchange/@tomgag/111815723861443432
- Ferret7446 1y agoThis "problem" is self-contradicting. If you can distinguish AI content, then you can just do that. If you can't, what's the problem?