7 ms·

GenAI, the snake eating its own tail

99 points by brikis98 9mo ago

mrcwinn 9mo ago
Pay per crawl of StackOverflow wouldn't encourage me to post more on StackOverflow. (Not that I was anyway.) Presumably you'd need to pay content creators, but that seems quite inefficient: 1. I pay OpenAI 2. OpenAI rev shares to StackOverflow 3. StackOverflow mostly keeps that money, but shares some with me for posting 4. I get some money back to help pay OpenAI? This is nonsense. And if the frontier labs are right about simulated data, as Tesla seems to have been right with its FSD simulated visualization stack, does this really matter anyway? The value I get from an LLM far exceeds anything I have ever received from SO or an O'Reilly book (as much as I genuinely enjoy them collecting dust on a shelf). If the argument is "fairness," I can sympathize but then shrug. If the argument is sustainability of training, I'm skeptical we need these payment models. And if the argument is about total value creation, I just don't buy it at all.
- lbrito 9mo ago
  >If the argument is sustainability of training, I'm skeptical we need these payment models. That seems to be the argument: LLM adoption leads to drop of organic training data, leading LLMs to eventually plateau, and we'll be left without the user-generated content we relied on for a while (like SO) and with subpar LLM. That's what I'm getting from the article anyway.
  - mapontosevenths 9mo ago
    The article gets the part about organic data dying off right. Look at Google SERP's for an example. Almost nobody clicks through to the source anymore, so ad revenue is drying up for them and people are publishing less or publishing in places that pay them directly and live behind a paywall like Medium. Which means Google has less data to work with. That said, what it misses is that the AI prompts themselves become a giant source of data. None of these companies are promising not to use your data, and even if you don't opt-in the person you sent the document/email/whatever to will because they want it paraphrased or need help understanding it.
    - lbrito 9mo ago
      >AI prompts themselves become a giant source of data. Good point, but can it match the old organic data? I'm skeptical. For one, the LLM environment lacks any truth or consensus mechanism that the old SO-like sites had. 100s of users might have discussed the same/similar technical problem with an LLM, but there's no way (afaik) for the AI to promote good content and demote bad ones, as it (AI) doesn't have the concept of correctness/truth. Also, the old sites were two-sided, with humans asking _and_ answering questions, while they are only on the asking side with AI.
      - cthalupa 9mo ago
        > 100s of users might have discussed the same/similar technical problem with an LLM, but there's no way (afaik) for the AI to promote good content and demote bad ones, as it (AI) doesn't have the concept of correctness/truth The LLM doesn't but reinforcement does. If someone keeps asking the model how to fix the problem after being given an answer, the answer is likely wrong. If someone deletes the chat after getting the answer, it was probably right.
      - mapontosevenths 8mo ago
        > (AI) doesn't have the concept of correctness/truth They kind of do, and it's getting better every day. We already have huge swatches of verifiable facts available to them to ground their statements in truth. They started building Cyc in 1984, and Wikipedia just signed deals with all the major players. The problem you're describing isn't intractable, so it's fairly certain that someone will solve it soon. Most of the brightest minds in society are working on AI in some form now. It's starting to sound trite, but today's AI's really are the worst that AI will ever be.
        dailycardriver 8mo ago
        “ Most of the brightest minds in society are working on AI in some form now.” Source? I haven’t met one intelligent person working on AI. The smartest people are being ground into dust. They’re being replaced by pompous overconfident people such as yourself.
  - TeMPOraL 9mo ago
    There are so many things wrong with the points this article repeats, but those are soundbites at this point so I'm not sure one can even argue against them anymore. Still, for the one about organic data (or "pre-war steel") drying out, it's not a threat to model development at all. People repeating this point don't realize that we already have way more data than we need. We got to where we are by brute-forcing the problem - throwing more data at a simple training process. If new "pristine" data were to stop flowing now, we still a) have decent pre-trained base models, and a dataset that's more than sufficient to train more of them, and b) lots of low-hanging fruits to pick in training approaches, architectures and data curation, that will allow to get more performance out of same base data. That, and the fact that synthetic data turned out to be quite effective after all, especially in the latter phases of training. No surprise there, for many classes of problems this is how we learn as well. Anyone who has experience studying math for maturity exam / university entry exams knows this: the best way to learn is to solve lots of variations of the same set of problems. These variations are all synthetic data, until recently generated by hand, but even their trivial nature doesn't make them less effective at teaching.
    - pixl97 8mo ago
      >We got to where we are by brute-forcing the problem This has been a bit of a concern of mine. That we have to do things the hard way for a long time, and in doing so make a massive amount of fast hardware. Then we get some breakthru that massively drops the amount of compute necessary, the surplus we suddenly have may lead to some kind of AI capability explosion.
- zzzeek 9mo ago
  > If the argument is sustainability of training, that is the argument, yes. Claude clearly got an enormous amount of its content from Stackoverflow. Which has mostly ceased to be a source of new content. However unlike the author I dont see any way to fix this; stackoverflow was only there because people had technical questions that needed answers. Maybe if the LLMs do indeed start going stale as there's not enough training data for new technologies, Q&A sites like Stackoverflow would still have a place, since people would still resort to asking each other questions rather than LLMs that dont have training data for a newer technology.
znsksjjs 9mo ago
We’ve seen decades of growing wage gaps and erosion of labors strength. The current elites don’t really care to enrich the people. Why would they care to do anything about this problem? They likely don’t see it as a problem at all. If they did actually stumble on AGI (assuming it didn’t eat them too) it would be used by a select few to enslave or remove the rest of us.
- alfalfasprout 9mo ago
  Not sure why this is being downvoted. It's spot on. You see folks like Dario et al. raising the alarm bells about what they claim is coming... while working as hard as they can to bring that gloomy future to fruition. No one in power is going to help unless there's money in it.
  - thatguy0900 9mo ago
    You can also see all of these people building survival bunkers.
  - lbrito 9mo ago
    Its being downvoted because HN has a very active billionaire-techbro-fanbase. Also who's this Dario?
    - mapontosevenths 9mo ago
      It's being downvoted because it's a ridiculous premise. "The Elites" are human too. This attitude is nonsensical and child-like. Nobody is out here trying to round up the hippies and force them to live in some kind of pods to be harvested for their nutrients or whatever. This technology, like every prior technology, will cause some people to lose their jobs and some new jobs to be created. This will annoy people who have to learn new skill instead of coasting until retirement as they planned. It is no different than the buggy whip manufacturers being annoyed at Henry Ford. They were right that it was bad for their industry, but wrong about it being the death of... well all the million things they claimed it would be the death of.
furyofantares 9mo ago
The article feels very confused to me. Example 1 is bad, StackOverflow had clearly plateaued and was well into the downward freefall by the time ChatGPT was released. Example 2 is apparently "open source" but it's actually just Tailwind which unfortunately had a very susceptible business model. And I don't really think the framing here that it's eating its own tail makes sense. It's also confusing to me why they're trying to solve the problem of it eating its own tail - there's a LOT of money being poured into the AI companies. They can try to solve that problem. What I mean is - a snake eating its own tail is bad for the snake. It will kill it. But in this case the tail is something we humans valued and don't want eaten, regardless of the health of the snake. And the snake will probably find a way to become independent of the tail after it ate it, rather than die, which sucks for us if we valued the stuff the tail was made of, and of course makes the analogy totally nonsensical. The actual solutions suggested here are not related to it eating its own tail anyway. They're related to the sentiment that the greed of AI companies needs to be reeled in, they need to give back, and we need solutions to the fact that we're getting spammed with slop. I guess the last part is the part that ties into it "eating its own tail", but really, why frame it that way? Framing it that way means it's a problem for AI companies. Let's be honest and say it's a problem for us and we want it solved for our own reasons.
- npinsker 9mo ago
  “Well, Reddit is growing, which contradicts my point, but I really feel like it’s not”
  - mapontosevenths 9mo ago
    Reddit is growing because they introduced automatic machine translation and Indians have been joining at an increasing rate. That content is mixed into the English language content, but is of very low quality and irrelevant to many native English speakers. Similarly they mix the English content in with the Indian content. Essentially, Reddit is also eating it's own tail to survive as the flood of low quality irrelevant content is making the platform worse for speakers of all languages but nobody cares because "line go up."
- 9mo ago
jaredcwhite 9mo ago
“We can’t put the genie back in the bottle.” Actually we can. And we will.
- semiquaver 9mo ago
  How?
  - happytoexplain 9mo ago
    Law or war. Not saying it would happen.
  - mapontosevenths 9mo ago
    It was presented without explanation and can be ignored without explanation.
    - jaredcwhite 9mo ago
      You need an explanation of how people make norms & laws regarding what is acceptable or unacceptable in society and industry?
      - mapontosevenths 9mo ago
        No such claim was made, therefore no such claim needs to be refuted. If people want to engage in conversation they will have to use their words to do it.
  - thatguy0900 9mo ago
    Only way I could see it is if there's enough pushback on them taking everyone's power and water (and computer parts) in a world where power and water are becoming increasingly unstable. But I feel like defeating Ai because there is not enough consistent water and power to give them means there is more pressing issues at hand...
  - atomic128 9mo ago
locusofself 9mo ago
I feel like the only solution to the problem is democratized RLHF, where whenever we get a bad answer from an LLM, we can immediately tell it what was wrong and it can learn from that.
- schmichael 9mo ago
  If you're paying to use the model that means instead of paying content creators you're also now giving more content to the model for free. Also just like SEO to game search engines, "democratized RLHF" has big trust issues.
- randomNumber7 8mo ago
  Maybe what is bad for you would be right for me.
aeon_ai 9mo ago
GenAI changes the dynamics of information systems so fundamentally that our entire notion of intellectual property is being upended. Copyright was predicated on the notion that ideas and styles can not be protected, but that explicit expressive works can. For example, a recipe can't be protected, but the story you wrap around it that tells how your grandma used to make it would be. LLMs are particularly challenging to wrangle with because they perform language alchemy. They can (and do) re-express the core ideas, styles, themes, etc. without violating copyright. People deem this 'theft' and 'stealing' because they are trying to reconcile the myth of intellectual property with reality, and are also simultaneously sensing the economic ladder being pulled up by elites who are watching and gaming the geopolitical world disorder. There will be a new system of value capture that content creators need to position for, which is to be seen as a more valuable source of high quality materials than an LLM, serving a specific market, and effectively acquiring attention to owned properties and products. It will not be pay-per-crawl. Or pay-per-use. It will be an attention game, just like everything in the modern economy. Attention is the only way you can monetize information.
- bitwize 8mo ago
  No. The idea-expression dichotomy is a common myth about copyright law, right up there with "if I already own the physical cartridge, downloading this game ROM is OK". The ONLY things that matter when determining whether copyright was infringed are "access" and "substantial similarity". The first refers to whether the alleged infringer did, or had a reasonable opportunity to, view the copyrighted work. The second is more vague and open-ended. But if these two, alone, can be established in court, then absent a fair use or other defense (for example, all of the ways in which your work is "substantially similar" to the infringed work are public domain), you are infringing. Period. End of story. The Tetris Company, for example, owns the idea of falling-tetromino puzzle video games. If you develop and release such a game, they will sue you and they will win. They have won in the past and they can retain Boies-tier lawyers to litigate a small crater where you once stood if need be. In fact, the ruling in the Tetris vs. Xio case means that look-and-feel copyrights, thought dead after Apple v. Microsoft and Lotus v. Borland, are now back on the table. It's not like this is even terribly new. Atari, license holders to Pac-Man on game consoles at the time, sued Philips over the release of K.C. Munchkin! on their rival console, the Magnavox Odyssey 2. Munchkin didn't look like Pac-Man. The monsters didn't look like the ghosts from Pac-Man. The mazes and some of the game mechanics were significantly different. Yet, the judge ruled that because it featured an "eater" who ate dots and avoided enemies in a maze, and sometimes had the opportunity to eat the enemies, K.C. Munchkin! infringed on the copyrights to Pac-Man. The ideas used in Pac-Man were novel enough to be eligible for copyright protection.
cthalupa 9mo ago
This is an article that I agreed with more reading the headline than I did when I finished reading the article itself. Stack Overflow peaked in 2014 before beginning it's downward decline. How is that at all related to GenAI? GPT4 is when we really started seeing these things get used to replace SO, etc., and that would be early 2023 - and indeed the drop gets worse there - but after the COVID era spike, SO was already crashing hard. Tailwind's business model was providing a component library built on top of their framework. It's a business model that relies on the framework being good enough for people to want to use it to begin with, but being bad enough that they'd rather pay for the component library than build it themselves. The more comfortable it is to use, the more productive it is, the worse the value proposition is for the premium upsell. Even other "open core" business models don't have this inherent dichotomy, much less open source on the whole, so it's really weird to try and extrapolate this out. The thing is, people turn to LLMs to solve problems and answer questions. If they can't turn to the LLM to solve that problem or answer that question, they'll either turn elsewhere, in which case there is still a market for that book or blog post, or they'll drop the problem and question and move on. And if they were willing to drop the problem or question and move on without investigating post-LLM, were they ever invested enough to buy your book, or check more than the first couple of results on google?
- worik 8mo ago
  I feel no nostalgia for Stackoverflow. I always found it very frustrating that for a person at the start of the learning curve it was "read only" Actually asking a naive question there was to get horribly flamed on the site. It, and the people using it, were very keen to explain how stupid you were being LLMs on the other hand are sweet and welcoming (to a fault) of the naive newbie I have been learning to use Shell script with the help of LLMs, I could not achieve that using SO Good riddance
  - georgefrowny 8mo ago
    It was very strange to me how StackOverflow consistently, every time it was mentioned everyone had exactly the same complaints and nothing changed. They can't have been unaware of the reputation or the sinking activity rates.
birdiefm 9mo ago
ouroboros can have a little ouroboros (as a treat)
WagesUpPriceDn 8mo ago
[dead]
jarjoura 8mo ago
This is exactly the sentiment I have been trying to articulate myself. The ONLY reason we are here today is because OpenAI, and Anthropic, by extension, took it upon themselves to launch chat bots trained on whatever datasources they could get in a short amount of time to quickly productize their investments. Their first versions didn't include any references to the source material, and just acted as if they knew everything. When CoPilot was built as a better auto-complete engine, trained on opensource projects, it was an interesting idea, because it doing what people already did. They searched GitHub for examples of the solution or nudged them in that direction. However, the biggest difference, using other project code was stable, because it came with a LICENSE.md that you then agreed to, and paid it forward. (i.e. "I used code from this project"). CoPilot initially would just inject snippets for you, without you knowing the source. It was only later, they walked that back and if you did use CoPilot, it shows you the most-likely source of the code it used. This is exactly the direction all of the platforms seem headed. It's not easy to walk back the free-for-all system (i.e. Napster), but I'm optimistic over time it'll become a more fair, pay to access system.
worik 8mo ago
The old model of selling eyeballs to advertisers was horrible I do not know what will replace it, but I will not miss websites trying to monetise my attention
- loudmax 8mo ago
  The GenAI providers will certainly explore advertisement revenue. They're not doing much of it yet because they're trying to gain market share while they figure out what what pain threshold of advertising their users will tolerate. People today may have a better sense of the downsides of ad-based services than we did when the internet was becoming mainstream. Back then, the minor inconvenience of seeing a few ads seemed worth all the benefits of access all the internet had to offer. And it probably was. But today the public has more experience with the downsides of relentless advertising optimization and audience capture, so there might be more business models based on something other than advertising. Either way, GenAI advertising is certainly coming.
  - 3vidence 8mo ago
    There has been a very public announcement that OpenAI will start exploring ads in their models https://openai.com/index/our-approach-to-advertising-and-expanding-access/ https://openai.com/index/our-approach-to-advertising-and-exp...
  - apercu 8mo ago
    "Back then, the minor inconvenience of seeing a few ads seemed worth all the benefits of access all the internet had to offer" Yes and no. I complained about ads to a partner back in 1999. He seemed surprised, and said something to the effect of "well that's how the content you are consuming is paid for". My argument (25 years ago now) was that it wasn't the ads as much as it was the ads blocking content, slowing the page load, being intrusive, etc. So, even "Back Then" it was an issue. Now, it's on steroids with all the aforementioned behaviours being even worse now (e.g., reload an entire page on mobile in order to load a new ad, pages jumping around as new ads are loaded, more pop-ups, etc.) but exacerbated by the privacy nightmare of weaponized data collection. I'm not disagreeing with you, I just think the underlying issues were evident very early in the "www" environment.
aogaili 8mo ago
Companies will provide incentives for people to generate authentic content. Example X giving $1M reward for the top Articles. Because they can use it for training. In fact, this might be overall good thing, because finally original content will be highly on demand since those companies now use to train their models. But we are probably just in a transition phase. The other thing is that new sources of input will come, from LLM usage probably, so they cut the middle layer, users input in the LLM is also a form of input, and a hybrid co-creation between users/AI would generate content at much faster rater, which again would be used to train the model, and that would improve their quality.
- randomNumber7 8mo ago
  How do you evaluate if content is authentic?
  - asdff 8mo ago
    Eventually it won’t matter. People will forget what it was like to have authentic content. They’d look at it like how we might view monks in the middle ages copying the bible a hundred times by hand. Even today people don’t even trust themselves to write an email. You see people on HN and reddit openly admitting they used ai to help make their post because they believe they cannot write. The march to illiteracy and ignorance is already taking foot.
    - hyperadvanced 8mo ago
      This is a plain fact. The amount of obviously GPT’d text out there is so breathtaking. The good news is that all of this shit people are squeezing out of GPT is really reinvigorating my desire to work on novel creative projects. In nearly every form of entertainment over the past 2 years, quality has degraded rapidly, to the point where most Reddit threads really aren’t worth reading - they’re all chock full of It’s Not X It’s Y! Most movies are heavily re-explained plots and reboots of old IP, CGI, dialogue glued on after the fact. Pop music has been on life support for 10 or 20 years, and that was before anyone could AI generate whatever derivative sonic slop they wanted to create. Books and publishers are holding strong to avoid GPT, but the desire for Dragon smut will ultimately overwhelm any aesthete’s preference for originality.
mikestorrent 8mo ago
> he took a PDF of my book Terraform: Up & Running, uploaded it into a GenAI tool, and asked the tool to follow the guidance in the book to generate Terraform code This is ridiculous - AI doesn't need to be fed a PDF of a Terraform book to know how to Terraform. Blowing out context with hundreds of OCR'd pages of generic text on how to terraform isn't going to help anything. The model that is broken is really ultimately going to be "content for hire". That's the industry that is going to be destroyed here because it's simply redundant now. Actual artwork, actual literature, actual music... these things are all safe as long as people actually want to experience the creations of others. Corporate artwork, simple documentation, elevator music.... these things are done; I'm sorry if you made a living making them but you were ultimately performing an artisinal task in a mostly soulless way. I'm not talking about video game artists, mind you, I'm talking about the people who produced Corporate Memphis and Flat Design here. We'll all be better off if these people find a new calling in life.
- pixl97 8mo ago
  >I'm not talking about video game artists You are talking about some video game artists, and while not their fault directly, EA pushing out SportsBall 2026 for the 40th year in a row is just a soulless corporate money printing machine.
Imustaskforhelp 8mo ago
Ironic that they created this post and I immediately felt like referencing ouroborous even when I hadn't looked at the website but then I opened it and the first thing I see is an AI generated image of Ouroborous Like the irony is pretty deep with this one about this. I am not sure if they could've gotten trademark from Inscryption/if they needed it but if they really wanted, I have found inscryption's ouroborous card to look the best and it was honestly how I discovered ouroborous in the first place! (became my favourite card, I love inscryption) https://static1.thegamerimages.com/wordpress/wp-content/uploads/2023/08/inscryption-card-ouroboros.jpg https://static1.thegamerimages.com/wordpress/wp-content/uplo... Even just searching Ouroborous on internet gave me some genuinely beautiful Ouroborous illustrations (Some stock photos, some not) but even using a stock photo might have made a better idea than using AI generated Ouroboros photo itself?
- tines 8mo ago
  Inscryption is the GOAT, beautiful game. Played it when I had Covid.
  - Imustaskforhelp 8mo ago
    > Inscryption is the GOAT, beautiful game. Played it when I had Covid. Oh Man, Y'know I once beat Inscryption's kaycee's mods challenge without doing any blood sacrifice the whole run after I got bored when I had defeated all kaycee's mods The gecko deck was amazing. I even played some mods of Inscryption. It was so lovely. The people who play inscryption & people who don't aren't the same lol. Played it during covid/end of it too & I still sing its Let's discuss some Inscryption, tell me your favourite card combo. Mine was ouroborous + the sigil which made it so that whenever a card died you got it back in hand, so now what you got was lets say you have ouroborous and some other card, you get ouroborous card placed lets say on the deck and then you get a squirrel and you have a wolf you place the squirrel and then kill the ouroborous and squirrel to get wolf but now ouroborous gets back to your hand with better stats for the whole game, then it becomes so strong that you just need a magpie card or such sigil to then look for ouroborous card to make it game over. Combining Ouroborous with the cockroach sigil + the three blood sigil essentially is an unlimited win game if you put the cockroach sigil on the blood goat too. Gecko/Stoat are my second best favourite. before doing puzzles you could stack as many sigils as possible and I had the best stoat in the whole world lol, had like 4-5 sigils iirc :) What about ya! There were also some really fascinating mechanics in the mods that I played too, definitely worth a play!
ropable 8mo ago
Everyone here throwing shade at Stack Overflow is clearly too young to remember the horror of Experts Exchange and every other technical help site prior to SO. For nearly a decade, it was absolutely transformative as a technical help resource. It certainly had its faults, but it was so far ahead of the other options as to be game-changing. I believe that the main reason for SO's decline starting around 2018 was that most of the core technical questions had been answered. There was an enormous existing corpus of accepted answers around fundamental topics, and technology just doesn't change fast enough to sustain the site. Then the LLMs digested the site's (beautifully machine-readable) corpus along with the rest of the internet and now the AIs can give users that info directly, resulting in a downward spiral of traffic to SO, fewer new questions, etc. Vale, Stack Overflow. You helped me solve many tricky problems.
- go_photon_go 8mo ago
  Stack Overflow should pivot to be an AI agents/local LLM information board and advertise the non-stack overflow stack exchange sites. There is a lot they could do.
  - deleted 8mo ago
    [deleted]
- ralph84 8mo ago
  Prior to SO we had Usenet, mailing lists, and IRC. They weren't so bad before spammers found them.
  - BrouteMinou 8mo ago
    Funny enough, you think that irc is dead, like most people, and the spammers... but let me tell you a little secret here:
- sharadov 8mo ago
  At one point I was in the top 10 in the Experts Exchange Leaderboard! It sucked as a platform, but I did learn a lot helping answer questions.
sharadov 8mo ago
Sounds a lot like how the original web was envisioned as a 2 way street, where people were paid for their "attention". Rather we became the product.
nubg 8mo ago
Isn't this a self solving problem? If LLMs won't be able to solve a certain problem, arbitrage opportunities open to solve it as a human (by writing a blog post for example), just like in the old days, people typically did this for fame. I think LLMs can be understood as a really good way to search existing solutions, including combining multiple solutions in the fly. Only if something cannot be "found", we go back to the dynamics of before LLMs.
- direwolf20 8mo ago
  There's zero fame for writing a blog post that's ingested by LLMs and adds to their knowledge base so they answer the question correctly next time.
- falloutx 8mo ago
  There may like 0.001% of the general population who would ever write a blog post, most of the LLM data in future would be fake blog posts used to trick LLMs or just spam.
cadamsdotcom 8mo ago
These takes are gonna age like milk. The new world is one where someone can have an LLM assisted insight, post it on their blog for free, have it indexed by every agentic search engine, and it becomes part of the zeitgeist. That’s the new data that’ll feed the new models: a better information diet over time. And guess what else: models are getting better at identifying - at scale - the high quality info that’s worth using as training data.
- nicbou 8mo ago
  But what would be the point? There is a huge difference between giving eggs to your neighbours and sending them to a breakfast restaurant in another state. A connection is made between you and the neighbors. A community forms around it. What would be the point of having a machine write your thoughts so they can shift a model's weights by infinitesimal amounts? How would that compare to building a small following, getting reader mail, and maybe even meeting a few of them? My website is used as training data. I get nothing from it. The AI twists words I have carefully selected, misleads people, and strips me of the fruit of my labour. If I lost my entire audience, I would stop doing that work. Moreover, there is a cost to producing high quality information. Research alone takes a long time when you're putting information online for the first time. I wouldn't do it for a vague chance of affecting the Zeitgeist.
pdyc 8mo ago
Strange nobody provided counterpoint. I would argue that questions meant for stackoverflow have shifted to genai tools and they have even richer data, questions, background of user and accepted answer so they dont need so data to improve. Side-effect of this is whoever gets most users wins and continues to win leading to monopoly as data is not public
josefritzishere 8mo ago
AI solves few problems that humans can't already solve, and creates systemic problems for which there are no solution. It's poison.
- apercu 8mo ago
  I have a lot of opinions about LLMs, and most are not positive due to the error rate, the security nightmare, the IP issues, the ecological impact and the hype, hype, hype that stands in direct opposition from my years of experiments (starting in 2019). But, I have to admit that a few years ago LLMs became part of my daily workflow for low-risk repetitive tasks, and sometimes even "sound boarding".
turnsout 8mo ago
Foundation model sponsors already pay humans to generate authentic content, especially in technical areas that are underrepresented in general internet scrapes. I would imagine that this trend will continue. Further, the "model collapse" hypothesis of 2020/2021 seems to have failed to materialize. Maybe we're still too early, and we're not yet seeing negative effects of OpenAI training on OpenAI output. But maybe "slop" is not being rewarded as much as human content, and having humans in the loop (even as readers) is preventing a slide into incoherence. Will LLMs eventually disincentivize people from producing and publishing new original content? If that content is easily replicated by an LLM query, maybe. And maybe it's not the worst thing in the world. 5 years ago I would have bought an "FFmpeg Cookbook" from O'Reilly, but now I would just tell Claude exactly what I'm trying to achieve. As a consumer, I'm better off, and arguably we've saved the author of a hypothetical FFmpeg Cookbook weeks out of their precious life. Weeks they could spend doing something—anything—more valuable than rewording FFmpeg documentation.
thestructuralme 8mo ago
The “snake eating its own tail” frame is real, but it’s not mystical — it’s incentives + sampling. If the web gets flooded with LLM output and you train on it naively, you’re effectively training on your own prior. That pushes models toward the mean: less surprise, less specificity, more template-y phrasing. It’s like photocopying a photocopy: the sharp edges disappear. The fix isn’t “never use synthetic data.” It’s to treat it like a controlled ingredient: tag provenance, keep a high-quality human/grounded core, filter aggressively, and anchor training to things that don’t self-contaminate (code that compiles/tests, math with verifiable proofs, retrieval with citations, real user feedback). Otherwise the easiest path is content volume, and volume is exactly what kills signal.
- iwontberude 8mo ago
  LLMs will always be just a little too random or a little too average. There in is the hidden beauty of AI: elevating the trust in peoples diverse experiences. Humans are amazing machines that reduce insane amounts of complexity in bespoke combinations of neural processors to synthesize ideas and emotions. Even Ilya Sutskever has said that he wasn't and still isn't clear at a formal level why GPT works at all (e.g. interpretability problem), but GPT was not a random discovery, it was based on work that was an amalgamation of Ilya and others careers and biases.

99 points by brikis98 9mo ago

Pay per crawl of StackOverflow wouldn't encourage me to post more on StackOverflow. (Not that I was anyway.) Presumably you'd need to pay content creators, but that seems quite inefficient: 1. I pay OpenAI 2. OpenAI rev shares to StackOverflow 3. StackOverflow mostly keeps that money, but shares some with me for posting 4. I get some money back to help pay OpenAI? This is nonsense. And if the frontier labs are right about simulated data, as Tesla seems to have been right with its FSD simulated visualization stack, does this really matter anyway? The value I get from an LLM far exceeds anything I have ever received from SO or an O'Reilly book (as much as I genuinely enjoy them collecting dust on a shelf). If the argument is "fairness," I can sympathize but then shrug. If the argument is sustainability of training, I'm skeptical we need these payment models. And if the argument is about total value creation, I just don't buy it at all.

lbrito 9mo ago
>If the argument is sustainability of training, I'm skeptical we need these payment models. That seems to be the argument: LLM adoption leads to drop of organic training data, leading LLMs to eventually plateau, and we'll be left without the user-generated content we relied on for a while (like SO) and with subpar LLM. That's what I'm getting from the article anyway.
- mapontosevenths 9mo ago
  The article gets the part about organic data dying off right. Look at Google SERP's for an example. Almost nobody clicks through to the source anymore, so ad revenue is drying up for them and people are publishing less or publishing in places that pay them directly and live behind a paywall like Medium. Which means Google has less data to work with. That said, what it misses is that the AI prompts themselves become a giant source of data. None of these companies are promising not to use your data, and even if you don't opt-in the person you sent the document/email/whatever to will because they want it paraphrased or need help understanding it.
  - lbrito 9mo ago
    >AI prompts themselves become a giant source of data. Good point, but can it match the old organic data? I'm skeptical. For one, the LLM environment lacks any truth or consensus mechanism that the old SO-like sites had. 100s of users might have discussed the same/similar technical problem with an LLM, but there's no way (afaik) for the AI to promote good content and demote bad ones, as it (AI) doesn't have the concept of correctness/truth. Also, the old sites were two-sided, with humans asking _and_ answering questions, while they are only on the asking side with AI.
    - cthalupa 9mo ago
      > 100s of users might have discussed the same/similar technical problem with an LLM, but there's no way (afaik) for the AI to promote good content and demote bad ones, as it (AI) doesn't have the concept of correctness/truth The LLM doesn't but reinforcement does. If someone keeps asking the model how to fix the problem after being given an answer, the answer is likely wrong. If someone deletes the chat after getting the answer, it was probably right.
    - mapontosevenths 8mo ago
      > (AI) doesn't have the concept of correctness/truth They kind of do, and it's getting better every day. We already have huge swatches of verifiable facts available to them to ground their statements in truth. They started building Cyc in 1984, and Wikipedia just signed deals with all the major players. The problem you're describing isn't intractable, so it's fairly certain that someone will solve it soon. Most of the brightest minds in society are working on AI in some form now. It's starting to sound trite, but today's AI's really are the worst that AI will ever be.
      - dailycardriver 8mo ago
        “ Most of the brightest minds in society are working on AI in some form now.” Source? I haven’t met one intelligent person working on AI. The smartest people are being ground into dust. They’re being replaced by pompous overconfident people such as yourself.
- TeMPOraL 9mo ago
  There are so many things wrong with the points this article repeats, but those are soundbites at this point so I'm not sure one can even argue against them anymore. Still, for the one about organic data (or "pre-war steel") drying out, it's not a threat to model development at all. People repeating this point don't realize that we already have way more data than we need. We got to where we are by brute-forcing the problem - throwing more data at a simple training process. If new "pristine" data were to stop flowing now, we still a) have decent pre-trained base models, and a dataset that's more than sufficient to train more of them, and b) lots of low-hanging fruits to pick in training approaches, architectures and data curation, that will allow to get more performance out of same base data. That, and the fact that synthetic data turned out to be quite effective after all, especially in the latter phases of training. No surprise there, for many classes of problems this is how we learn as well. Anyone who has experience studying math for maturity exam / university entry exams knows this: the best way to learn is to solve lots of variations of the same set of problems. These variations are all synthetic data, until recently generated by hand, but even their trivial nature doesn't make them less effective at teaching.
  - pixl97 8mo ago
    >We got to where we are by brute-forcing the problem This has been a bit of a concern of mine. That we have to do things the hard way for a long time, and in doing so make a massive amount of fast hardware. Then we get some breakthru that massively drops the amount of compute necessary, the surplus we suddenly have may lead to some kind of AI capability explosion.
zzzeek 9mo ago
> If the argument is sustainability of training, that is the argument, yes. Claude clearly got an enormous amount of its content from Stackoverflow. Which has mostly ceased to be a source of new content. However unlike the author I dont see any way to fix this; stackoverflow was only there because people had technical questions that needed answers. Maybe if the LLMs do indeed start going stale as there's not enough training data for new technologies, Q&A sites like Stackoverflow would still have a place, since people would still resort to asking each other questions rather than LLMs that dont have training data for a newer technology.

znsksjjs 9mo ago

We’ve seen decades of growing wage gaps and erosion of labors strength. The current elites don’t really care to enrich the people. Why would they care to do anything about this problem? They likely don’t see it as a problem at all. If they did actually stumble on AGI (assuming it didn’t eat them too) it would be used by a select few to enslave or remove the rest of us.

alfalfasprout 9mo ago
Not sure why this is being downvoted. It's spot on. You see folks like Dario et al. raising the alarm bells about what they claim is coming... while working as hard as they can to bring that gloomy future to fruition. No one in power is going to help unless there's money in it.
- thatguy0900 9mo ago
  You can also see all of these people building survival bunkers.
- lbrito 9mo ago
  Its being downvoted because HN has a very active billionaire-techbro-fanbase. Also who's this Dario?
  - mapontosevenths 9mo ago
    It's being downvoted because it's a ridiculous premise. "The Elites" are human too. This attitude is nonsensical and child-like. Nobody is out here trying to round up the hippies and force them to live in some kind of pods to be harvested for their nutrients or whatever. This technology, like every prior technology, will cause some people to lose their jobs and some new jobs to be created. This will annoy people who have to learn new skill instead of coasting until retirement as they planned. It is no different than the buggy whip manufacturers being annoyed at Henry Ford. They were right that it was bad for their industry, but wrong about it being the death of... well all the million things they claimed it would be the death of.

furyofantares 9mo ago

The article feels very confused to me. Example 1 is bad, StackOverflow had clearly plateaued and was well into the downward freefall by the time ChatGPT was released. Example 2 is apparently "open source" but it's actually just Tailwind which unfortunately had a very susceptible business model. And I don't really think the framing here that it's eating its own tail makes sense. It's also confusing to me why they're trying to solve the problem of it eating its own tail - there's a LOT of money being poured into the AI companies. They can try to solve that problem. What I mean is - a snake eating its own tail is bad for the snake. It will kill it. But in this case the tail is something we humans valued and don't want eaten, regardless of the health of the snake. And the snake will probably find a way to become independent of the tail after it ate it, rather than die, which sucks for us if we valued the stuff the tail was made of, and of course makes the analogy totally nonsensical. The actual solutions suggested here are not related to it eating its own tail anyway. They're related to the sentiment that the greed of AI companies needs to be reeled in, they need to give back, and we need solutions to the fact that we're getting spammed with slop. I guess the last part is the part that ties into it "eating its own tail", but really, why frame it that way? Framing it that way means it's a problem for AI companies. Let's be honest and say it's a problem for us and we want it solved for our own reasons.

npinsker 9mo ago
“Well, Reddit is growing, which contradicts my point, but I really feel like it’s not”
- mapontosevenths 9mo ago
  Reddit is growing because they introduced automatic machine translation and Indians have been joining at an increasing rate. That content is mixed into the English language content, but is of very low quality and irrelevant to many native English speakers. Similarly they mix the English content in with the Indian content. Essentially, Reddit is also eating it's own tail to survive as the flood of low quality irrelevant content is making the platform worse for speakers of all languages but nobody cares because "line go up."
9mo ago

jaredcwhite 9mo ago

“We can’t put the genie back in the bottle.” Actually we can. And we will.

semiquaver 9mo ago
How?
- happytoexplain 9mo ago
  Law or war. Not saying it would happen.
- mapontosevenths 9mo ago
  It was presented without explanation and can be ignored without explanation.
  - jaredcwhite 9mo ago
    You need an explanation of how people make norms & laws regarding what is acceptable or unacceptable in society and industry?
    - mapontosevenths 9mo ago
      No such claim was made, therefore no such claim needs to be refuted. If people want to engage in conversation they will have to use their words to do it.
- thatguy0900 9mo ago
  Only way I could see it is if there's enough pushback on them taking everyone's power and water (and computer parts) in a world where power and water are becoming increasingly unstable. But I feel like defeating Ai because there is not enough consistent water and power to give them means there is more pressing issues at hand...
- atomic128 9mo ago

locusofself 9mo ago

I feel like the only solution to the problem is democratized RLHF, where whenever we get a bad answer from an LLM, we can immediately tell it what was wrong and it can learn from that.

schmichael 9mo ago
If you're paying to use the model that means instead of paying content creators you're also now giving more content to the model for free. Also just like SEO to game search engines, "democratized RLHF" has big trust issues.
randomNumber7 8mo ago
Maybe what is bad for you would be right for me.

aeon_ai 9mo ago

GenAI changes the dynamics of information systems so fundamentally that our entire notion of intellectual property is being upended. Copyright was predicated on the notion that ideas and styles can not be protected, but that explicit expressive works can. For example, a recipe can't be protected, but the story you wrap around it that tells how your grandma used to make it would be. LLMs are particularly challenging to wrangle with because they perform language alchemy. They can (and do) re-express the core ideas, styles, themes, etc. without violating copyright. People deem this 'theft' and 'stealing' because they are trying to reconcile the myth of intellectual property with reality, and are also simultaneously sensing the economic ladder being pulled up by elites who are watching and gaming the geopolitical world disorder. There will be a new system of value capture that content creators need to position for, which is to be seen as a more valuable source of high quality materials than an LLM, serving a specific market, and effectively acquiring attention to owned properties and products. It will not be pay-per-crawl. Or pay-per-use. It will be an attention game, just like everything in the modern economy. Attention is the only way you can monetize information.

bitwize 8mo ago
No. The idea-expression dichotomy is a common myth about copyright law, right up there with "if I already own the physical cartridge, downloading this game ROM is OK". The ONLY things that matter when determining whether copyright was infringed are "access" and "substantial similarity". The first refers to whether the alleged infringer did, or had a reasonable opportunity to, view the copyrighted work. The second is more vague and open-ended. But if these two, alone, can be established in court, then absent a fair use or other defense (for example, all of the ways in which your work is "substantially similar" to the infringed work are public domain), you are infringing. Period. End of story. The Tetris Company, for example, owns the idea of falling-tetromino puzzle video games. If you develop and release such a game, they will sue you and they will win. They have won in the past and they can retain Boies-tier lawyers to litigate a small crater where you once stood if need be. In fact, the ruling in the Tetris vs. Xio case means that look-and-feel copyrights, thought dead after Apple v. Microsoft and Lotus v. Borland, are now back on the table. It's not like this is even terribly new. Atari, license holders to Pac-Man on game consoles at the time, sued Philips over the release of K.C. Munchkin! on their rival console, the Magnavox Odyssey 2. Munchkin didn't look like Pac-Man. The monsters didn't look like the ghosts from Pac-Man. The mazes and some of the game mechanics were significantly different. Yet, the judge ruled that because it featured an "eater" who ate dots and avoided enemies in a maze, and sometimes had the opportunity to eat the enemies, K.C. Munchkin! infringed on the copyrights to Pac-Man. The ideas used in Pac-Man were novel enough to be eligible for copyright protection.

cthalupa 9mo ago

This is an article that I agreed with more reading the headline than I did when I finished reading the article itself. Stack Overflow peaked in 2014 before beginning it's downward decline. How is that at all related to GenAI? GPT4 is when we really started seeing these things get used to replace SO, etc., and that would be early 2023 - and indeed the drop gets worse there - but after the COVID era spike, SO was already crashing hard. Tailwind's business model was providing a component library built on top of their framework. It's a business model that relies on the framework being good enough for people to want to use it to begin with, but being bad enough that they'd rather pay for the component library than build it themselves. The more comfortable it is to use, the more productive it is, the worse the value proposition is for the premium upsell. Even other "open core" business models don't have this inherent dichotomy, much less open source on the whole, so it's really weird to try and extrapolate this out. The thing is, people turn to LLMs to solve problems and answer questions. If they can't turn to the LLM to solve that problem or answer that question, they'll either turn elsewhere, in which case there is still a market for that book or blog post, or they'll drop the problem and question and move on. And if they were willing to drop the problem or question and move on without investigating post-LLM, were they ever invested enough to buy your book, or check more than the first couple of results on google?

worik 8mo ago
I feel no nostalgia for Stackoverflow. I always found it very frustrating that for a person at the start of the learning curve it was "read only" Actually asking a naive question there was to get horribly flamed on the site. It, and the people using it, were very keen to explain how stupid you were being LLMs on the other hand are sweet and welcoming (to a fault) of the naive newbie I have been learning to use Shell script with the help of LLMs, I could not achieve that using SO Good riddance
- georgefrowny 8mo ago
  It was very strange to me how StackOverflow consistently, every time it was mentioned everyone had exactly the same complaints and nothing changed. They can't have been unaware of the reputation or the sinking activity rates.

birdiefm 9mo ago

ouroboros can have a little ouroboros (as a treat)

WagesUpPriceDn 8mo ago

[dead]

jarjoura 8mo ago

This is exactly the sentiment I have been trying to articulate myself. The ONLY reason we are here today is because OpenAI, and Anthropic, by extension, took it upon themselves to launch chat bots trained on whatever datasources they could get in a short amount of time to quickly productize their investments. Their first versions didn't include any references to the source material, and just acted as if they knew everything. When CoPilot was built as a better auto-complete engine, trained on opensource projects, it was an interesting idea, because it doing what people already did. They searched GitHub for examples of the solution or nudged them in that direction. However, the biggest difference, using other project code was stable, because it came with a LICENSE.md that you then agreed to, and paid it forward. (i.e. "I used code from this project"). CoPilot initially would just inject snippets for you, without you knowing the source. It was only later, they walked that back and if you did use CoPilot, it shows you the most-likely source of the code it used. This is exactly the direction all of the platforms seem headed. It's not easy to walk back the free-for-all system (i.e. Napster), but I'm optimistic over time it'll become a more fair, pay to access system.

worik 8mo ago

The old model of selling eyeballs to advertisers was horrible I do not know what will replace it, but I will not miss websites trying to monetise my attention

loudmax 8mo ago
The GenAI providers will certainly explore advertisement revenue. They're not doing much of it yet because they're trying to gain market share while they figure out what what pain threshold of advertising their users will tolerate. People today may have a better sense of the downsides of ad-based services than we did when the internet was becoming mainstream. Back then, the minor inconvenience of seeing a few ads seemed worth all the benefits of access all the internet had to offer. And it probably was. But today the public has more experience with the downsides of relentless advertising optimization and audience capture, so there might be more business models based on something other than advertising. Either way, GenAI advertising is certainly coming.
- 3vidence 8mo ago
  There has been a very public announcement that OpenAI will start exploring ads in their models https://openai.com/index/our-approach-to-advertising-and-expanding-access/ https://openai.com/index/our-approach-to-advertising-and-exp...
- apercu 8mo ago
  "Back then, the minor inconvenience of seeing a few ads seemed worth all the benefits of access all the internet had to offer" Yes and no. I complained about ads to a partner back in 1999. He seemed surprised, and said something to the effect of "well that's how the content you are consuming is paid for". My argument (25 years ago now) was that it wasn't the ads as much as it was the ads blocking content, slowing the page load, being intrusive, etc. So, even "Back Then" it was an issue. Now, it's on steroids with all the aforementioned behaviours being even worse now (e.g., reload an entire page on mobile in order to load a new ad, pages jumping around as new ads are loaded, more pop-ups, etc.) but exacerbated by the privacy nightmare of weaponized data collection. I'm not disagreeing with you, I just think the underlying issues were evident very early in the "www" environment.

aogaili 8mo ago

Companies will provide incentives for people to generate authentic content. Example X giving $1M reward for the top Articles. Because they can use it for training. In fact, this might be overall good thing, because finally original content will be highly on demand since those companies now use to train their models. But we are probably just in a transition phase. The other thing is that new sources of input will come, from LLM usage probably, so they cut the middle layer, users input in the LLM is also a form of input, and a hybrid co-creation between users/AI would generate content at much faster rater, which again would be used to train the model, and that would improve their quality.

randomNumber7 8mo ago
How do you evaluate if content is authentic?
- asdff 8mo ago
  Eventually it won’t matter. People will forget what it was like to have authentic content. They’d look at it like how we might view monks in the middle ages copying the bible a hundred times by hand. Even today people don’t even trust themselves to write an email. You see people on HN and reddit openly admitting they used ai to help make their post because they believe they cannot write. The march to illiteracy and ignorance is already taking foot.
  - hyperadvanced 8mo ago
    This is a plain fact. The amount of obviously GPT’d text out there is so breathtaking. The good news is that all of this shit people are squeezing out of GPT is really reinvigorating my desire to work on novel creative projects. In nearly every form of entertainment over the past 2 years, quality has degraded rapidly, to the point where most Reddit threads really aren’t worth reading - they’re all chock full of It’s Not X It’s Y! Most movies are heavily re-explained plots and reboots of old IP, CGI, dialogue glued on after the fact. Pop music has been on life support for 10 or 20 years, and that was before anyone could AI generate whatever derivative sonic slop they wanted to create. Books and publishers are holding strong to avoid GPT, but the desire for Dragon smut will ultimately overwhelm any aesthete’s preference for originality.

mikestorrent 8mo ago

> he took a PDF of my book Terraform: Up & Running, uploaded it into a GenAI tool, and asked the tool to follow the guidance in the book to generate Terraform code This is ridiculous - AI doesn't need to be fed a PDF of a Terraform book to know how to Terraform. Blowing out context with hundreds of OCR'd pages of generic text on how to terraform isn't going to help anything. The model that is broken is really ultimately going to be "content for hire". That's the industry that is going to be destroyed here because it's simply redundant now. Actual artwork, actual literature, actual music... these things are all safe as long as people actually want to experience the creations of others. Corporate artwork, simple documentation, elevator music.... these things are done; I'm sorry if you made a living making them but you were ultimately performing an artisinal task in a mostly soulless way. I'm not talking about video game artists, mind you, I'm talking about the people who produced Corporate Memphis and Flat Design here. We'll all be better off if these people find a new calling in life.

pixl97 8mo ago
>I'm not talking about video game artists You are talking about some video game artists, and while not their fault directly, EA pushing out SportsBall 2026 for the 40th year in a row is just a soulless corporate money printing machine.

Imustaskforhelp 8mo ago

Ironic that they created this post and I immediately felt like referencing ouroborous even when I hadn't looked at the website but then I opened it and the first thing I see is an AI generated image of Ouroborous Like the irony is pretty deep with this one about this. I am not sure if they could've gotten trademark from Inscryption/if they needed it but if they really wanted, I have found inscryption's ouroborous card to look the best and it was honestly how I discovered ouroborous in the first place! (became my favourite card, I love inscryption) https://static1.thegamerimages.com/wordpress/wp-content/uploads/2023/08/inscryption-card-ouroboros.jpg https://static1.thegamerimages.com/wordpress/wp-content/uplo... Even just searching Ouroborous on internet gave me some genuinely beautiful Ouroborous illustrations (Some stock photos, some not) but even using a stock photo might have made a better idea than using AI generated Ouroboros photo itself?

tines 8mo ago
Inscryption is the GOAT, beautiful game. Played it when I had Covid.
- Imustaskforhelp 8mo ago
  > Inscryption is the GOAT, beautiful game. Played it when I had Covid. Oh Man, Y'know I once beat Inscryption's kaycee's mods challenge without doing any blood sacrifice the whole run after I got bored when I had defeated all kaycee's mods The gecko deck was amazing. I even played some mods of Inscryption. It was so lovely. The people who play inscryption & people who don't aren't the same lol. Played it during covid/end of it too & I still sing its Let's discuss some Inscryption, tell me your favourite card combo. Mine was ouroborous + the sigil which made it so that whenever a card died you got it back in hand, so now what you got was lets say you have ouroborous and some other card, you get ouroborous card placed lets say on the deck and then you get a squirrel and you have a wolf you place the squirrel and then kill the ouroborous and squirrel to get wolf but now ouroborous gets back to your hand with better stats for the whole game, then it becomes so strong that you just need a magpie card or such sigil to then look for ouroborous card to make it game over. Combining Ouroborous with the cockroach sigil + the three blood sigil essentially is an unlimited win game if you put the cockroach sigil on the blood goat too. Gecko/Stoat are my second best favourite. before doing puzzles you could stack as many sigils as possible and I had the best stoat in the whole world lol, had like 4-5 sigils iirc :) What about ya! There were also some really fascinating mechanics in the mods that I played too, definitely worth a play!

ropable 8mo ago

Everyone here throwing shade at Stack Overflow is clearly too young to remember the horror of Experts Exchange and every other technical help site prior to SO. For nearly a decade, it was absolutely transformative as a technical help resource. It certainly had its faults, but it was so far ahead of the other options as to be game-changing. I believe that the main reason for SO's decline starting around 2018 was that most of the core technical questions had been answered. There was an enormous existing corpus of accepted answers around fundamental topics, and technology just doesn't change fast enough to sustain the site. Then the LLMs digested the site's (beautifully machine-readable) corpus along with the rest of the internet and now the AIs can give users that info directly, resulting in a downward spiral of traffic to SO, fewer new questions, etc. Vale, Stack Overflow. You helped me solve many tricky problems.

go_photon_go 8mo ago
Stack Overflow should pivot to be an AI agents/local LLM information board and advertise the non-stack overflow stack exchange sites. There is a lot they could do.
- deleted 8mo ago
  [deleted]
ralph84 8mo ago
Prior to SO we had Usenet, mailing lists, and IRC. They weren't so bad before spammers found them.
- BrouteMinou 8mo ago
  Funny enough, you think that irc is dead, like most people, and the spammers... but let me tell you a little secret here:
sharadov 8mo ago
At one point I was in the top 10 in the Experts Exchange Leaderboard! It sucked as a platform, but I did learn a lot helping answer questions.

sharadov 8mo ago

Sounds a lot like how the original web was envisioned as a 2 way street, where people were paid for their "attention". Rather we became the product.

nubg 8mo ago

Isn't this a self solving problem? If LLMs won't be able to solve a certain problem, arbitrage opportunities open to solve it as a human (by writing a blog post for example), just like in the old days, people typically did this for fame. I think LLMs can be understood as a really good way to search existing solutions, including combining multiple solutions in the fly. Only if something cannot be "found", we go back to the dynamics of before LLMs.

direwolf20 8mo ago
There's zero fame for writing a blog post that's ingested by LLMs and adds to their knowledge base so they answer the question correctly next time.
falloutx 8mo ago
There may like 0.001% of the general population who would ever write a blog post, most of the LLM data in future would be fake blog posts used to trick LLMs or just spam.

cadamsdotcom 8mo ago

These takes are gonna age like milk. The new world is one where someone can have an LLM assisted insight, post it on their blog for free, have it indexed by every agentic search engine, and it becomes part of the zeitgeist. That’s the new data that’ll feed the new models: a better information diet over time. And guess what else: models are getting better at identifying - at scale - the high quality info that’s worth using as training data.

nicbou 8mo ago
But what would be the point? There is a huge difference between giving eggs to your neighbours and sending them to a breakfast restaurant in another state. A connection is made between you and the neighbors. A community forms around it. What would be the point of having a machine write your thoughts so they can shift a model's weights by infinitesimal amounts? How would that compare to building a small following, getting reader mail, and maybe even meeting a few of them? My website is used as training data. I get nothing from it. The AI twists words I have carefully selected, misleads people, and strips me of the fruit of my labour. If I lost my entire audience, I would stop doing that work. Moreover, there is a cost to producing high quality information. Research alone takes a long time when you're putting information online for the first time. I wouldn't do it for a vague chance of affecting the Zeitgeist.

pdyc 8mo ago

Strange nobody provided counterpoint. I would argue that questions meant for stackoverflow have shifted to genai tools and they have even richer data, questions, background of user and accepted answer so they dont need so data to improve. Side-effect of this is whoever gets most users wins and continues to win leading to monopoly as data is not public

josefritzishere 8mo ago

AI solves few problems that humans can't already solve, and creates systemic problems for which there are no solution. It's poison.

apercu 8mo ago
I have a lot of opinions about LLMs, and most are not positive due to the error rate, the security nightmare, the IP issues, the ecological impact and the hype, hype, hype that stands in direct opposition from my years of experiments (starting in 2019). But, I have to admit that a few years ago LLMs became part of my daily workflow for low-risk repetitive tasks, and sometimes even "sound boarding".

turnsout 8mo ago

Foundation model sponsors already pay humans to generate authentic content, especially in technical areas that are underrepresented in general internet scrapes. I would imagine that this trend will continue. Further, the "model collapse" hypothesis of 2020/2021 seems to have failed to materialize. Maybe we're still too early, and we're not yet seeing negative effects of OpenAI training on OpenAI output. But maybe "slop" is not being rewarded as much as human content, and having humans in the loop (even as readers) is preventing a slide into incoherence. Will LLMs eventually disincentivize people from producing and publishing new original content? If that content is easily replicated by an LLM query, maybe. And maybe it's not the worst thing in the world. 5 years ago I would have bought an "FFmpeg Cookbook" from O'Reilly, but now I would just tell Claude exactly what I'm trying to achieve. As a consumer, I'm better off, and arguably we've saved the author of a hypothetical FFmpeg Cookbook weeks out of their precious life. Weeks they could spend doing something—anything—more valuable than rewording FFmpeg documentation.

thestructuralme 8mo ago

The “snake eating its own tail” frame is real, but it’s not mystical — it’s incentives + sampling. If the web gets flooded with LLM output and you train on it naively, you’re effectively training on your own prior. That pushes models toward the mean: less surprise, less specificity, more template-y phrasing. It’s like photocopying a photocopy: the sharp edges disappear. The fix isn’t “never use synthetic data.” It’s to treat it like a controlled ingredient: tag provenance, keep a high-quality human/grounded core, filter aggressively, and anchor training to things that don’t self-contaminate (code that compiles/tests, math with verifiable proofs, retrieval with citations, real user feedback). Otherwise the easiest path is content volume, and volume is exactly what kills signal.

iwontberude 8mo ago
LLMs will always be just a little too random or a little too average. There in is the hidden beauty of AI: elevating the trust in peoples diverse experiences. Humans are amazing machines that reduce insane amounts of complexity in bespoke combinations of neural processors to synthesize ideas and emotions. Even Ilya Sutskever has said that he wasn't and still isn't clear at a formal level why GPT works at all (e.g. interpretability problem), but GPT was not a random discovery, it was based on work that was an amalgamation of Ilya and others careers and biases.