5 ms·

> No, other people did. They wrote about it, and LLM can sometimes use that. Once they no longer write about it, what then? It can read the code? Historical di

> No, other people did. They wrote about it, and LLM can sometimes use that. Once they no longer write about it, what then? It can read the code? Historical discussions around it? Commit histories? > But even then, people aren't entitled to the knowledge "created" by doing the work. If attribution and compensation were tackled in earnest, if you could only train on the materials of the people you pay to produce those materials, it might be much quicker and cheaper to just learn CSS. OSS code and people’s public writings are available to anyone all the time. Common Crawl, the open source web crawl dump, has been around for over a decade. No one had any problem with these systems being developed on them, until they finally started to become useful, so what’s the sort of legal or ethical framework you’re pointing to?

oblio 5mo ago
> It can read the code? Historical discussions around it? Commit histories? And if everyone bunkers up and all that open content dries up starting in 2026, let's say, what happens?
- kristianc 5mo ago
  It won't happen, for two reasons. One is that great deal of open-source software and hobbyist knowledge sharing has never been driven by financial reward anyway and people will continue to do it anyway. Finer grained controls over opt-outs would be great (the equivalent of a search engine 'nofollow' would be great and will hopefully come with time). Many kinds of technology faced this kind of tragedy of the commons argument in the past and it never bears out. Printing presses copied manuscripts, search engines copied and indexed web pages, open-source software was incorporated into commercial products, Wikipedia repackaged knowledge produced elsewhere. In almost all cases the total amount of creation increases because the technology lowered costs, expanded audiences, or created new forms of value. The speed of creation of new 'View Source' outpaces the number of people pulling back.
  - customguy 5mo ago
    > In almost all cases the total amount of creation increases because the technology lowered costs But this doesn't lower the cost of learning and writing CSS, it just scoops up some of it and offers that cheaply, and even that only because it's offered below cost. If anything I'd say it increases the cost, because now you don't get paid to get and be good at what an LLM is supposedly good enough at, and have less free time to do it anyway. You may not even have a computer because your current one broke and you can't afford a new one.
  - watwut 5mo ago
    It will happen and it already started to happen. It started to happen even before LLM, when google started to hide smaller personal blogs in its search result. Expectation of the monetary reward has nothing to do with it, discoverability does. Culture of creating content does not exist when people cant see what others created and know no one will see what they created. A lot of smaller open source was monkey see monkey do thing - we have seen other open source projects and wanted something like that. Likewise with tutorials, we have seen other people write cool tutorials and felt like creating own and showing it out. That is not the dynamic with LLM. You see LLM output, but original creator is hidden. And if you write your own, no one will find it. Worst, other people will tell you "LLM could have write it" in reaction ... so people wont bother. > search engines copied and indexed web pages Notably, search engines sent people toward web pages. And when search engines stopped doing that and started to copy content, those original pages started to die out. > Printing presses copied manuscripts Printing press made dissemination easier. It is an equivalent of early internet, not of LLM. > open-source software was incorporated into commercial products Commercial product using open source library had different user then the library it is using. And crucially, it is not hiding that library from the library user. > Wikipedia repackaged knowledge produced elsewhere Yes, and we collectively create less encyclopedias. They are not worth writing and checking for correctness anymore, so we don't do that all that much anymore.
    - visarga 5mo ago
      The centralized choke point of web search is getting relaxed now. Unlike search engines and social networks, you can download a LLM and run it. A small one, but capable of using a library of search stubs to directly fetch information from hubs, feeds and other search engines. You can own the agent who can solve the web search part for you. Imagine you have a 4B model and keep an equal size corpus of search stubs, small MD documents linking to hubs, feeds and search engines for millions of topics. You can use the LLM to read the stub and perform the search for you, all orchestrated locally, with greater privacy and independence. You can dis-intermediate the search chokepoint now. You can set the criteria for what to include, exclude, how to rank and present the results. This works because good entry points for any topic change slowly over time. The construction of search stubs is trivial with existing AI agents, and can be shared as open source. A few GB for the model, a few for the search routing layer, and you got a sovereign local agent. If this holds, access control shifts from whatever Google thinks maximizes profit to whatever the community thinks has value.
- aspenmartin 5mo ago
  Well that historical content and code still exists right? Are you just saying “what if we’re in a world of walled gardens now that OSS dies because people don’t want their work stolen” in which case: these companies will get data and they don’t need OSS anymore. It’s already webcrawled or licensed or commissioned, they pay people to generate novel traces when they need it or at the very least sets of prompts and tests for verification. Then synthetic data gets added to the training set, the ones that are verified.
  - oblio 5mo ago
    This is super hilarious :-))) Do you think creating the orders of magnitude of content the internet produced organically and which LLM creators are stealing is cheap? If they actually have to pay for content creation while competing with content creators on the you know, content creation front via LLM-generation, the entire business model of LLMs collapses. You can't have the mountains of data needed for LLMs in the decades to come, if your LLMs put the writers and artists out of work.
    - aspenmartin 5mo ago
      It’s literally how these models are trained today. They of course use open source data but that’s no longer the most important source, it’s high quality prompts and verifiable tests and a lot of inference compute. They also have massive flywheels from users from which they can mine good data or at the very least again good prompts which can be just as important.
      - oblio 5mo ago
        And everything we know about these companies points to unsustainability, before we even get to very high impact content lawsuits which haven't even been settled. Let alone lots of data sets being pulled out of public view and being moved to anti LLM licenses (with explicit licensing for training). We will see how this shakes out in the coming years, as Anthropic, OpenAI & co file for IPO or run out of private funding. Grok is already on the ropes as seen from the SpaceX IPO.
customguy 5mo ago
> It can read the code? Historical discussions around it? Commit histories? Assume everybody is now using LLM because they're better, and because the people who created artisanal things in their free time out of sheer generosity no longer have free time, or any food at all, or simply no longer feel generous. And the few people who are such specialists that they would be slowed down by them only do proprietary work, for lots of money. What then? LLM learning from LLM doesn't really work, does it? This is not intended as some kind of gotcha, to me this is a huge elephant on the couch. > No one had any problem with these systems being developed on them, until they finally started to become useful, so what’s the sort of legal or ethical framework you’re pointing to? That it's perfectly fine for people to say "I was fine with that, but I'm not fine with this". They can give you detailed explanations for their individual decisions, every single one of them, but there is no point in discussing them in aggregate because that aggregate is an abstraction. And they're optional, too, it's not like people have to give an explanation, and aren't simply free to change their mind for no or for bad reasons.
- aspenmartin 5mo ago
  > Assume everybody is now using LLM because they're better, and because the people who created artisanal things in their free time out of sheer generosity no longer have free time, or any food at all, or simply no longer feel generous. And the few people who are such specialists that they would be slowed down by them only do proprietary work, for lots of money. > What then? LLM learning from LLM doesn't really work, does it? Oh what no that’s exactly how it works, even today. RL with verification is done with synthetic data and rejection sampling. If something can’t get done purely with an agent that needs to get done it’s done with human help, this will always be the case it will just get rare-er. > That it's perfectly fine for people to say "I was fine with that, but I'm not fine with this". Agree with you there, but there’s a theme or insinuation (not saying you’re saying this) that these companies “stole work” (which definitely a lot of copyright violations sure), but it’s just unclear to me what principles or legal frameworks these companies or institutions should have used to develop the technology. I don’t really even know whether I mean to imply it’s not unethical, moreso I’m looking for a steel man argument to this. But of course people are entitled to their value systems and judgements and to point out real harm.

oblio 5mo ago

> It can read the code? Historical discussions around it? Commit histories? And if everyone bunkers up and all that open content dries up starting in 2026, let's say, what happens?

kristianc 5mo ago
It won't happen, for two reasons. One is that great deal of open-source software and hobbyist knowledge sharing has never been driven by financial reward anyway and people will continue to do it anyway. Finer grained controls over opt-outs would be great (the equivalent of a search engine 'nofollow' would be great and will hopefully come with time). Many kinds of technology faced this kind of tragedy of the commons argument in the past and it never bears out. Printing presses copied manuscripts, search engines copied and indexed web pages, open-source software was incorporated into commercial products, Wikipedia repackaged knowledge produced elsewhere. In almost all cases the total amount of creation increases because the technology lowered costs, expanded audiences, or created new forms of value. The speed of creation of new 'View Source' outpaces the number of people pulling back.
- customguy 5mo ago
  > In almost all cases the total amount of creation increases because the technology lowered costs But this doesn't lower the cost of learning and writing CSS, it just scoops up some of it and offers that cheaply, and even that only because it's offered below cost. If anything I'd say it increases the cost, because now you don't get paid to get and be good at what an LLM is supposedly good enough at, and have less free time to do it anyway. You may not even have a computer because your current one broke and you can't afford a new one.
- watwut 5mo ago
  It will happen and it already started to happen. It started to happen even before LLM, when google started to hide smaller personal blogs in its search result. Expectation of the monetary reward has nothing to do with it, discoverability does. Culture of creating content does not exist when people cant see what others created and know no one will see what they created. A lot of smaller open source was monkey see monkey do thing - we have seen other open source projects and wanted something like that. Likewise with tutorials, we have seen other people write cool tutorials and felt like creating own and showing it out. That is not the dynamic with LLM. You see LLM output, but original creator is hidden. And if you write your own, no one will find it. Worst, other people will tell you "LLM could have write it" in reaction ... so people wont bother. > search engines copied and indexed web pages Notably, search engines sent people toward web pages. And when search engines stopped doing that and started to copy content, those original pages started to die out. > Printing presses copied manuscripts Printing press made dissemination easier. It is an equivalent of early internet, not of LLM. > open-source software was incorporated into commercial products Commercial product using open source library had different user then the library it is using. And crucially, it is not hiding that library from the library user. > Wikipedia repackaged knowledge produced elsewhere Yes, and we collectively create less encyclopedias. They are not worth writing and checking for correctness anymore, so we don't do that all that much anymore.
  - visarga 5mo ago
    The centralized choke point of web search is getting relaxed now. Unlike search engines and social networks, you can download a LLM and run it. A small one, but capable of using a library of search stubs to directly fetch information from hubs, feeds and other search engines. You can own the agent who can solve the web search part for you. Imagine you have a 4B model and keep an equal size corpus of search stubs, small MD documents linking to hubs, feeds and search engines for millions of topics. You can use the LLM to read the stub and perform the search for you, all orchestrated locally, with greater privacy and independence. You can dis-intermediate the search chokepoint now. You can set the criteria for what to include, exclude, how to rank and present the results. This works because good entry points for any topic change slowly over time. The construction of search stubs is trivial with existing AI agents, and can be shared as open source. A few GB for the model, a few for the search routing layer, and you got a sovereign local agent. If this holds, access control shifts from whatever Google thinks maximizes profit to whatever the community thinks has value.
aspenmartin 5mo ago
Well that historical content and code still exists right? Are you just saying “what if we’re in a world of walled gardens now that OSS dies because people don’t want their work stolen” in which case: these companies will get data and they don’t need OSS anymore. It’s already webcrawled or licensed or commissioned, they pay people to generate novel traces when they need it or at the very least sets of prompts and tests for verification. Then synthetic data gets added to the training set, the ones that are verified.
- oblio 5mo ago
  This is super hilarious :-))) Do you think creating the orders of magnitude of content the internet produced organically and which LLM creators are stealing is cheap? If they actually have to pay for content creation while competing with content creators on the you know, content creation front via LLM-generation, the entire business model of LLMs collapses. You can't have the mountains of data needed for LLMs in the decades to come, if your LLMs put the writers and artists out of work.
  - aspenmartin 5mo ago
    It’s literally how these models are trained today. They of course use open source data but that’s no longer the most important source, it’s high quality prompts and verifiable tests and a lot of inference compute. They also have massive flywheels from users from which they can mine good data or at the very least again good prompts which can be just as important.
    - oblio 5mo ago
      And everything we know about these companies points to unsustainability, before we even get to very high impact content lawsuits which haven't even been settled. Let alone lots of data sets being pulled out of public view and being moved to anti LLM licenses (with explicit licensing for training). We will see how this shakes out in the coming years, as Anthropic, OpenAI & co file for IPO or run out of private funding. Grok is already on the ropes as seen from the SpaceX IPO.

customguy 5mo ago

> It can read the code? Historical discussions around it? Commit histories? Assume everybody is now using LLM because they're better, and because the people who created artisanal things in their free time out of sheer generosity no longer have free time, or any food at all, or simply no longer feel generous. And the few people who are such specialists that they would be slowed down by them only do proprietary work, for lots of money. What then? LLM learning from LLM doesn't really work, does it? This is not intended as some kind of gotcha, to me this is a huge elephant on the couch. > No one had any problem with these systems being developed on them, until they finally started to become useful, so what’s the sort of legal or ethical framework you’re pointing to? That it's perfectly fine for people to say "I was fine with that, but I'm not fine with this". They can give you detailed explanations for their individual decisions, every single one of them, but there is no point in discussing them in aggregate because that aggregate is an abstraction. And they're optional, too, it's not like people have to give an explanation, and aren't simply free to change their mind for no or for bad reasons.

aspenmartin 5mo ago
> Assume everybody is now using LLM because they're better, and because the people who created artisanal things in their free time out of sheer generosity no longer have free time, or any food at all, or simply no longer feel generous. And the few people who are such specialists that they would be slowed down by them only do proprietary work, for lots of money. > What then? LLM learning from LLM doesn't really work, does it? Oh what no that’s exactly how it works, even today. RL with verification is done with synthetic data and rejection sampling. If something can’t get done purely with an agent that needs to get done it’s done with human help, this will always be the case it will just get rare-er. > That it's perfectly fine for people to say "I was fine with that, but I'm not fine with this". Agree with you there, but there’s a theme or insinuation (not saying you’re saying this) that these companies “stole work” (which definitely a lot of copyright violations sure), but it’s just unclear to me what principles or legal frameworks these companies or institutions should have used to develop the technology. I don’t really even know whether I mean to imply it’s not unethical, moreso I’m looking for a steel man argument to this. But of course people are entitled to their value systems and judgements and to point out real harm.