6 ms·
somebody said once we are mining "low-background tokens" like we are mining low-background (radiation) steel post WW2 and i couldnt shake the concept out of my
by swyx 11mo ago
somebody said once we are mining "low-background tokens" like we are mining low-background (radiation) steel post WW2 and i couldnt shake the concept out of my head
(wrote up in https://www.latent.space/i/139368545/the-concept-of-low-background-tokens https://www.latent.space/i/139368545/the-concept-of-low-back... - but ironically repeating something somebody else said online is kinda what i'm willingly participating in, and it's unclear why human-origin tokens should be that much higher signal than ai-origin ones)
- jeffchuber 11mo agothat was me swyx
- rollulus 11mo agoMultiple people have coined the idea repeatedly, way before you. The oldest comment on HN I could find was in December 2022 by user spawarotti: https://news.ycombinator.com/item?id=33856172 https://news.ycombinator.com/item?id=33856172
- threeducks 11mo agoHere is an even older comment chain about it from 2020: https://news.ycombinator.com/item?id=23895706 https://news.ycombinator.com/item?id=23895706 Apparently, comparing low-background steel to pre-LLM text is a rather obvious analogy.
- rollulus 11mo agoOh wow, great find! That’s really early days.
- pseidemann 11mo agoAs well as that people often do think alike. If you have a thought, it's likely it's not new.
- jeffchuber 11mo agoi didnt claim to invent it. i claimed swyx heard it through me - which he did
- swyx 11mo agoyou did!!
- jrjfjgkrj 11mo agoevery human generation built upon the slop of the previous one but we appreciated that, we called it "standing on the shoulders of giants"
- rebuilder 11mo agoThat's because the things we built on weren't slop
- deleted 11mo ago[deleted]
- walrusted 11mo agothe only structure you can build with slop is a burial mound
- Dilettante_ 11mo agoWhat is unhardened concrete but slop?
- walrusted 11mo agoit makes the sound of slop, but the similarity effectively ends there. you cant build an approvable heavy load bearing beam from slop but you can build a burial mound from both.
- hoppp 11mo agoYou can't build on slop because slop is a slippery slope
- cindyllm 11mo ago[dead]
- Dilettante_ 11mo agoMaybe we'll have to slurp the slop so we don't slip on the slope.
- mwidell 11mo agoLow background steel is no longer necessary. "...began to fall in 1963, when the Partial Nuclear Test Ban Treaty was enacted, and by 2008 it had decreased to only 0.005 mSv/yr above natural levels. This has made special low-background steel no longer necessary for most radiation-sensitive uses, as new steel now has a low enough radioactive signature." https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel
- juvoly 11mo agoInteresting. I guess that analogously, we might find that X years after some future AI content production ban, we could similarly start ignoring the low background token issue?
- actionfromafar 11mo agoWe used a rather low number of atmospheric bombs, while we are carpet bombing the internet every day with AI marketing copy.
- MadnessASAP 11mo agoThe eternal September has finally ended. We've now entered the AI winter. It promises to be long, dark, and full of annoyances.
- embedding-shape 11mo ago"Winter" in AI (or cryptocurrency, or any at all) ecosystems denote a period of low activity, and a focus on fundamentals instead of driven by hype. What we're seeing now is something more like the peak of summer. If it ends up being a bubble, and it burtst, some months after that will be "AI Winter" as investors won't want to continue chucking money at problems anymore, and it'll go back to "in the background research" again, as it was before.
- 11mo ago
- alansaber 11mo agoSince synthetic data for training is pretty ubiquitous seems like a novelty