3 ms·

He's correct though. You would need 3 extremely low probability events to all occur simultaneously for the concern to actually manifest. 1) Courts reverse thei

He's correct though. You would need 3 extremely low probability events to all occur simultaneously for the concern to actually manifest. 1) Courts reverse their previous decisions and declare LLM generated code as belonging to LLM labs. 2) LLM labs decide to assert their copyright and sue open source projects. 3) They are able to prove that the code was generated by an LLM and not just any LLM but their LLM.

Retr0id 3mo ago
Isn't there also a concern that an LLM may reproduce copyrighted code verbatim (or close enough), and the original author asserts their copyright?
- worthless-trash 3mo ago
  I had early chatGPT spit my own erlang code back at me WITH my comments.
- justthehuman 3mo ago
  +1, I also thought this was the actual concern.
- olalonde 3mo ago
  This doesn't really happen by accident, especially where it's significant enough to constitute copyright infringement. And to my knowledge, there hasn't been a single lawsuit over this in the entire history of LLM-assisted coding.
  - dijksterhuis 3mo ago
    > there hasn't been a single lawsuit over this in the entire history of LLM-assisted coding. the "entire history of" is circa 3-4 years, which is very much a tiny period of time compared to normal legal system / copyright law stuff (IANAL).
    - olalonde 3mo ago
      It's enough to tell you that, if it does happen, it must be quite rare and not a widespread systemic issue. It's also worth noting that Anthropic and co are so confident that it's a non-issue that they offer full copyright indemnification to their commercial users.
      - dijksterhuis 3mo ago
        > It's enough to tell you that, if it does happen, it must be quite rare and not a systematic issue. alternative perspective: it's just taking time for the lawyers to figure out what they can sue them for.
        olalonde 3mo ago
        Not sure I understand. If some company or open source project is violating your copyright, you sue them for copyright infringement, no?
- bluGill 3mo ago
  There is concern, but the more I think about it, the less convinced I am. LLMs might produce a snippet, but copyright requires more than that. Last I checked (I'm not a lawyer, this was far enough back that the law has had major changes, and different countries have different laws! - read this with plenty of salt) if you start with copyright work and replace all but 10% that final amount just disappears since there isn't enough original to claim copyright anymore. You can never claim copyright over 'printf("Hello World");' - it is too generic. Most of what I've seen an LLM produce is likewise generic enough that even if it is a copy, it wouldn't be a copyrightable snippet.
  - dijksterhuis 3mo ago
    reminded me of: > The complaint argued that "the basis of the Gaye defendants' claims is that "Blurred Lines" and "Got To Give It Up" "feel" or "sound" the same. Being reminiscent of a "sound" is not copyright infringement. The intent in producing "Blurred Lines" was to evoke an era. In reality, the Gaye defendants are claiming ownership of an entire genre, as opposed to a specific work" they lost (eventually) https://en.wikipedia.org/wiki/Pharrell_Williams_v._Bridgeport_Music https://en.wikipedia.org/wiki/Pharrell_Williams_v._Bridgepor... wider point -- whether or not a copy is a copy and whether it is is infringing on copyright or not ultimately has to be decided by a court case when it's not an obvious and clear cut violation. especially in the USA with the utterly mental fair use law.
- user43928 3mo ago
  To my knowledge there is not a single known case where this happened. News back then were about intentionally prompting to output known copyrighted material. The parent comment still stands in my opinion: When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest?
  - jryan49 3mo ago
    I mean ai has generated copyright headers verbatim in code I asked it to make. I had a tutorial open from the web and asked the AI to generate an example for me and it was the example from the tutorial verbatim as well...
    - user43928 3mo ago
      This never happened to me and there is not a single such occurrence in my main project, across 150k LOC. I would be surprised if a frontier model generated unexpected copyright headers during typical usage.
      - nemomarx 3mo ago
        the people blindly submitting prs are not going to be using expensive frontier models, though.
      - jryan49 3mo ago
        It was a frontier model from a few months ago. It hasn't done it recently. But it has happened. I imagine if there isn't a lot of training data for an idea it probably has a higher change of generating verbatim, so as time goes on, it should become less of a problem (at least with high quality models).
  - dijksterhuis 3mo ago
    > To my knowledge there is not a single known case where this happened. > News back then were about intentionally prompting to output known copyrighted material. First, there are other cases if you take the time to dig. This is quite an old example (GPT-2) as i haven't kept up to date on this field recently, but it does show that this problem has been known about since before these systems were widely adopted: https://arxiv.org/abs/2012.07805 https://arxiv.org/abs/2012.07805 [0] Second, GP said nothing about the type of effort required to make it happen, just that it can be done and that the copyright owner could come along and cause legal problems later. It's absolutely possible to have a fly-by contributor who purposefully asks for code that reproduces X/Y/Z without a maintainer knowing about it. But then the maintainer is the one in legal trouble. > When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest? Legal / copyright / etc. cases often take a lot longer than a couple of years to come to fruition. --- [0]: edit -- to clarify this is an example of the reproduction problem, not an example copyright infringement case.
- agentultra 3mo ago
  That would be an example of, “overfitting.” It is something the AI labs are concerned about. They tweak parameters to try and avoid it. It’s a hard balancing act to do. Give in too much randomness and you get non-sensical outputs that are difficult to align. Fit too closely to the training data and the model regurgitates the training data. And oh, what’s that copyrighted material we never made any agreement to use doing in there?
- jmull 3mo ago
  Not to mention the moral concern. IP law (like a lot of other things) has been skewed toward the interests of business, even when that conflicts with fairness or societal good. For all its flaws (IMO), the free software movement tends to be principled. Just because something is legal doesn’t mean it’s right.
- panzi 3mo ago
  Not just when it reproduces code. Since fair use doesn't exist like that in a lot of countries the question is if the training itself is a violation. Are there e.g. EU or Japanese rulings on that?
luke5441 3mo ago
No? 1) Someone re-licenses GCC under a non-GPL license. 2) EFF sues them, to stop the behaviour 3) Court tells EFF that they have no standing to sue because LLM generated content has no copyright Obviously this happening would be in the future after someone translated GCC to Rust with LLMs or something.
- olalonde 3mo ago
  Again, it would require an extremely unlikely set of circumstances. 1) All of GCC would have to be LLM generated. If some parts are not, it's sufficient to prevent the re-licensing. 2) That someone would have to prove that all the GCC code was in fact LLM generated. Good luck doing that. 3) A court would have to decide that all the LLM-generated code does in fact fall under public domain, because it involved insufficient human input.
- skeledrew 3mo ago
  > Someone re-licenses GCC under a non-GPL license. A pointless act since code is now free. The GPL exists to ensure code freedom in an era when code was expensive. Yes I'm aware that the meaning of "free" is a bit conflated here, but the point stands.
  - lanstin 3mo ago
    I believe you are underestimating the value of GCC and LLVM having won the compiler wars and being extremely high quality, and of Linux having won the OS wars, being extremely high quality. I use GenAI every day, but the quality is still proportional to the amount of my time - there is a multiplier for quantity of my amount but the 9s approaching 1 quality is not logarithmically different - the effort to fix one big is lower, but the total effort to make excellent software is still a process of polishing and then iterating. The embedded quality in GCC will take a similar order of magnitude investment from an LLM rewrite. These super high quality open foundations make LLMs more useful to us now, but saying that the embedded value or cost of GCC or Linux is zero I think will steer people wrong.
tsimionescu 3mo ago
The concern related to copyright is not that the AI labs would assert copyright over the code produced by their LLMs - that is a complete strawman. The concern instead is that LLMs and all of their outputs may be found to be derivative works of their entire training set, and thus rendered unusable (as the training set is not distirbutable under any license). I think this ship has long sailed and no court is going to dare give such a decision given the money involved, for better or for worse. But it's a much more realistic scenario, in principle, than LLM labs going mad and attacking their own customers. Edit to add: there is another, completely different, copyright risk associated with LLMs - and one that is much more realistic. It is the fact that code generated by LLMs may not, in fact, be copyrightable at all. Which would mean that it can't be subject to the GPL. As long as it remains a minority of GCC code, this wouldn't matter much, but it could in time lead to significant portions of GCC becoming public domain, and thus cooyable, modifiable, and redistrubutable without providing the four freedoms.
- 7e 3mo ago
  In many timelines, non-copyrightable code in GPL project doesn’t restrict the user’s freedom, since an LLM can easily reimplement the functionality clean-room. AI has brought levels of freedom to users that GNU could never do. Practical levels of freedom, leading to real, positive outcomes. Software is finally free in the real world.
  - gspr 3mo ago
    Cleanroom?! What the hell kind of definition of a cleanroom is it when you get to take hundreds of GB of compressed code with you into it?!
  - tsimionescu 3mo ago
    For certain use cases, probably. For others, no. For example, hardware that uses GNU software must be patchable by users (if it is patchable at all). Hardware running public software can be completely locked down to its users, regardless of how powerful of an LLM the user has access to.
- olalonde 3mo ago
sodapopcan 3mo ago
> He's correct though. He's correct but it's an irrelevant argument, he's simply making an emotional, and very childish, attack on someone because they don't like the tech he likes. Banning LLMs in your project, regardless of the reason, does not mean you think they are ever going to go away, or even that you want them to.
archagon 3mo ago
You think that not a single powerful country in the world will get enough pissed-off people in power to successfully sue LLM labs for (blatant and willful) copyright infringement? Especially if they succeed in decimating the job market? How do you build a legally-sound product using an LLM that has been successfully sued for violating copyright in X countries around the world?

Retr0id 3mo ago

Isn't there also a concern that an LLM may reproduce copyrighted code verbatim (or close enough), and the original author asserts their copyright?

worthless-trash 3mo ago
I had early chatGPT spit my own erlang code back at me WITH my comments.
justthehuman 3mo ago
+1, I also thought this was the actual concern.
olalonde 3mo ago
This doesn't really happen by accident, especially where it's significant enough to constitute copyright infringement. And to my knowledge, there hasn't been a single lawsuit over this in the entire history of LLM-assisted coding.
- dijksterhuis 3mo ago
  > there hasn't been a single lawsuit over this in the entire history of LLM-assisted coding. the "entire history of" is circa 3-4 years, which is very much a tiny period of time compared to normal legal system / copyright law stuff (IANAL).
  - olalonde 3mo ago
    It's enough to tell you that, if it does happen, it must be quite rare and not a widespread systemic issue. It's also worth noting that Anthropic and co are so confident that it's a non-issue that they offer full copyright indemnification to their commercial users.
    - dijksterhuis 3mo ago
      > It's enough to tell you that, if it does happen, it must be quite rare and not a systematic issue. alternative perspective: it's just taking time for the lawyers to figure out what they can sue them for.
      - olalonde 3mo ago
        Not sure I understand. If some company or open source project is violating your copyright, you sue them for copyright infringement, no?
bluGill 3mo ago
There is concern, but the more I think about it, the less convinced I am. LLMs might produce a snippet, but copyright requires more than that. Last I checked (I'm not a lawyer, this was far enough back that the law has had major changes, and different countries have different laws! - read this with plenty of salt) if you start with copyright work and replace all but 10% that final amount just disappears since there isn't enough original to claim copyright anymore. You can never claim copyright over 'printf("Hello World");' - it is too generic. Most of what I've seen an LLM produce is likewise generic enough that even if it is a copy, it wouldn't be a copyrightable snippet.
- dijksterhuis 3mo ago
  reminded me of: > The complaint argued that "the basis of the Gaye defendants' claims is that "Blurred Lines" and "Got To Give It Up" "feel" or "sound" the same. Being reminiscent of a "sound" is not copyright infringement. The intent in producing "Blurred Lines" was to evoke an era. In reality, the Gaye defendants are claiming ownership of an entire genre, as opposed to a specific work" they lost (eventually) https://en.wikipedia.org/wiki/Pharrell_Williams_v._Bridgeport_Music https://en.wikipedia.org/wiki/Pharrell_Williams_v._Bridgepor... wider point -- whether or not a copy is a copy and whether it is is infringing on copyright or not ultimately has to be decided by a court case when it's not an obvious and clear cut violation. especially in the USA with the utterly mental fair use law.
user43928 3mo ago
To my knowledge there is not a single known case where this happened. News back then were about intentionally prompting to output known copyrighted material. The parent comment still stands in my opinion: When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest?
- jryan49 3mo ago
  I mean ai has generated copyright headers verbatim in code I asked it to make. I had a tutorial open from the web and asked the AI to generate an example for me and it was the example from the tutorial verbatim as well...
  - user43928 3mo ago
    This never happened to me and there is not a single such occurrence in my main project, across 150k LOC. I would be surprised if a frontier model generated unexpected copyright headers during typical usage.
    - nemomarx 3mo ago
      the people blindly submitting prs are not going to be using expensive frontier models, though.
    - jryan49 3mo ago
      It was a frontier model from a few months ago. It hasn't done it recently. But it has happened. I imagine if there isn't a lot of training data for an idea it probably has a higher change of generating verbatim, so as time goes on, it should become less of a problem (at least with high quality models).
- dijksterhuis 3mo ago
  > To my knowledge there is not a single known case where this happened. > News back then were about intentionally prompting to output known copyrighted material. First, there are other cases if you take the time to dig. This is quite an old example (GPT-2) as i haven't kept up to date on this field recently, but it does show that this problem has been known about since before these systems were widely adopted: https://arxiv.org/abs/2012.07805 https://arxiv.org/abs/2012.07805 [0] Second, GP said nothing about the type of effort required to make it happen, just that it can be done and that the copyright owner could come along and cause legal problems later. It's absolutely possible to have a fly-by contributor who purposefully asks for code that reproduces X/Y/Z without a maintainer knowing about it. But then the maintainer is the one in legal trouble. > When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest? Legal / copyright / etc. cases often take a lot longer than a couple of years to come to fruition. --- [0]: edit -- to clarify this is an example of the reproduction problem, not an example copyright infringement case.
agentultra 3mo ago
That would be an example of, “overfitting.” It is something the AI labs are concerned about. They tweak parameters to try and avoid it. It’s a hard balancing act to do. Give in too much randomness and you get non-sensical outputs that are difficult to align. Fit too closely to the training data and the model regurgitates the training data. And oh, what’s that copyrighted material we never made any agreement to use doing in there?
jmull 3mo ago
Not to mention the moral concern. IP law (like a lot of other things) has been skewed toward the interests of business, even when that conflicts with fairness or societal good. For all its flaws (IMO), the free software movement tends to be principled. Just because something is legal doesn’t mean it’s right.
panzi 3mo ago
Not just when it reproduces code. Since fair use doesn't exist like that in a lot of countries the question is if the training itself is a violation. Are there e.g. EU or Japanese rulings on that?

luke5441 3mo ago

No? 1) Someone re-licenses GCC under a non-GPL license. 2) EFF sues them, to stop the behaviour 3) Court tells EFF that they have no standing to sue because LLM generated content has no copyright Obviously this happening would be in the future after someone translated GCC to Rust with LLMs or something.

olalonde 3mo ago
Again, it would require an extremely unlikely set of circumstances. 1) All of GCC would have to be LLM generated. If some parts are not, it's sufficient to prevent the re-licensing. 2) That someone would have to prove that all the GCC code was in fact LLM generated. Good luck doing that. 3) A court would have to decide that all the LLM-generated code does in fact fall under public domain, because it involved insufficient human input.
skeledrew 3mo ago
> Someone re-licenses GCC under a non-GPL license. A pointless act since code is now free. The GPL exists to ensure code freedom in an era when code was expensive. Yes I'm aware that the meaning of "free" is a bit conflated here, but the point stands.
- lanstin 3mo ago
  I believe you are underestimating the value of GCC and LLVM having won the compiler wars and being extremely high quality, and of Linux having won the OS wars, being extremely high quality. I use GenAI every day, but the quality is still proportional to the amount of my time - there is a multiplier for quantity of my amount but the 9s approaching 1 quality is not logarithmically different - the effort to fix one big is lower, but the total effort to make excellent software is still a process of polishing and then iterating. The embedded quality in GCC will take a similar order of magnitude investment from an LLM rewrite. These super high quality open foundations make LLMs more useful to us now, but saying that the embedded value or cost of GCC or Linux is zero I think will steer people wrong.

tsimionescu 3mo ago

The concern related to copyright is not that the AI labs would assert copyright over the code produced by their LLMs - that is a complete strawman. The concern instead is that LLMs and all of their outputs may be found to be derivative works of their entire training set, and thus rendered unusable (as the training set is not distirbutable under any license). I think this ship has long sailed and no court is going to dare give such a decision given the money involved, for better or for worse. But it's a much more realistic scenario, in principle, than LLM labs going mad and attacking their own customers. Edit to add: there is another, completely different, copyright risk associated with LLMs - and one that is much more realistic. It is the fact that code generated by LLMs may not, in fact, be copyrightable at all. Which would mean that it can't be subject to the GPL. As long as it remains a minority of GCC code, this wouldn't matter much, but it could in time lead to significant portions of GCC becoming public domain, and thus cooyable, modifiable, and redistrubutable without providing the four freedoms.

7e 3mo ago
In many timelines, non-copyrightable code in GPL project doesn’t restrict the user’s freedom, since an LLM can easily reimplement the functionality clean-room. AI has brought levels of freedom to users that GNU could never do. Practical levels of freedom, leading to real, positive outcomes. Software is finally free in the real world.
- gspr 3mo ago
  Cleanroom?! What the hell kind of definition of a cleanroom is it when you get to take hundreds of GB of compressed code with you into it?!
- tsimionescu 3mo ago
  For certain use cases, probably. For others, no. For example, hardware that uses GNU software must be patchable by users (if it is patchable at all). Hardware running public software can be completely locked down to its users, regardless of how powerful of an LLM the user has access to.
olalonde 3mo ago

sodapopcan 3mo ago

> He's correct though. He's correct but it's an irrelevant argument, he's simply making an emotional, and very childish, attack on someone because they don't like the tech he likes. Banning LLMs in your project, regardless of the reason, does not mean you think they are ever going to go away, or even that you want them to.

archagon 3mo ago

You think that not a single powerful country in the world will get enough pissed-off people in power to successfully sue LLM labs for (blatant and willful) copyright infringement? Especially if they succeed in decimating the job market? How do you build a legally-sound product using an LLM that has been successfully sued for violating copyright in X countries around the world?