5 ms·

I still find the idea that "learning" from code is "stealing" kind of ridiculous.

by tensor 5mo ago

estimator7292 5mo ago
Learning, probably not. Copy/pasting at scale, yes
- vorticalbox 5mo ago
  It is learning though. It’s not just copying the code. Code gets turned into tokens and then it learns the next most likely token. The issue that I see most people talk about it the scale at which is learnt. A human will learn from other people’s code but not from every persons code.
  - cogman10 5mo ago
    The issue is that of copyright law WRT to derivative works. Machine transformations on original works does not create a new copyright for the person that directed the machine transformation. That's why you can't pirate a bunch of media by simply adding a red pixel to the righthand corner or by color shifting the video. Copyright law is very clear that if a machine does it, the original copyright on the input is kept. This is why your distributed binaries are still copyrighted, because the machine transformed, very significantly, the source code into binary which maintains the copyright throughout. It would be inconsistent for the courts to suddenly decide that "actually, this specific type of machine transformation is actually innovative." I know this is generally really bad for the AI industry, so they just ignore it until a court tells them they can't anymore. And they might get away with it as I don't have faith that the courts will be consistent.
    - red75prime 5mo ago
      Shredding is a machine transformation. Does it mean that shreds retain original copyright even if the content can't be restored and the provenance can't be traced? Just an example that treating all machine transformations equally with no regard to the specifics doesn't make much sense. And the specifics of autoregressive pretraining is that it is lossy compression. Good luck finding which copyrighted materials have made it into the final weights.
      - cogman10 5mo ago
        > Does it mean that shreds retain original copyright even if the content can't be restored? Yup, it absolutely does. In fact, that's why you are still violating copyright law by using bittorrent even though each of the users is only giving out a small slice or shred of the original content. The US has a granted defense in the case of something like shredding called "Fair Use" but that doesn't mean or imply that a copyright is void simply because of a fair use claim. > And the specifics of autoregressive pretraining is that it is lossy compression. That doesn't matter. Why would it? If I take a FLAC recording and change it to an MP3. The fact that it was a lossy transform doesn't suddenly give me the legal right to distribute the MP3. > Good luck finding which copyrighted materials have made it into the final weights. That's what the NYT v. OpenAI lawsuit is all about. And for earlier models they could, in fact, pull out full NYT articles which proved they made it into the final weights. Further, the NYT is currently in discovery which means OpenAI must open up to the NYT what goes into their weights. A move that, if OpenAI loses, other litigants can also use because there's a real good shot that OpenAI also included their works in the dataset.
        red75prime 5mo ago
        > Yup, it absolutely does Well, it's not the first time when the law contradicts laws of nature (for the entertainment of the future generations). Bittorent is not a relevant example, because the system is designed to restore the work in its fullness. > in fact, pull out full NYT articles That's when they used their knowledge of the exact text they wanted to "retrieve" to get the text? It wouldn't be so efficient with a random number generator, but it's doable.
        cogman10 5mo ago
        > Bittorent is not a relevant example, because the system is designed to restore the work in its fullness. You can restore shredded documents with enough time and effort. And if you did that and started making photo copies, even if they are incomplete, you will run afoul of copyright law. Bittorrent is a relevant example because it shows that shredding doesn't destroy copyright. Remember, copyright is about the right to copy something. Simply shredding or destroying a thing isn't applicable to copyright. Nor is giving that thing away. What's applicable is when you start to actually copy the thing.
  - blks 5mo ago
    A human is not a commercial product. Here we have commercial product that was created by using a lot of various copyrighted and protected IP, without licensing agreements, without paying, without even citing it.
  - ell1e 5mo ago
    LLMs seem to be so devoid of intelligence, I think it's arguable if that's learning: https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think... Typically, you would imply a level of understanding when you say learning. LLMs apparently can't do that, by design.
- margalabargala 5mo ago
  Copy/pasting at scale is how tons of software has been written for a long time, or have we all forgotten the jokes people used to make about StackOverflow?
pydry 5mo ago
If you can set a copyright trap and an LLM reproduces it I think it's pretty clear cut that it's more than just "learning". I have seen LLMs do all sorts of crap which was clearly reproduction of training material. This is also why people are most impressed with how much better it is at reproducing boilerplate rather than, say, imaginative new ideas.
- jakeydus 5mo ago
  Remember last year (?) when one of the major AIs produced a bit of code that included Jeff Geerling's name in a comment?
lo_zamoyski 5mo ago
If there were the case, then imagine having to give it back!
boh 5mo ago
Yes I guess there's also no such thing as stealing in torrents since the computer "learns" the data and returns it in a transcoded fashion so it's technically not a reproduction. Yes LLMs can reproduce passages from copyrighted works verbatim but that's only because it "learned" it and it's just telling you what it "knows". The mental calisthenics required to justify this stuff must be exhausting.
- idle_zealot 5mo ago
  > The mental calisthenics required to justify this stuff must be exhausting. It's only exhausting if you think copyright ever reasonably settled the matter of ownership of knowledge and want to morally justify an incoherent set of outcomes that they personally favor. In practice it's primarily been a tool for the powerful party in any dispute to hammer others for disrupting their business model. I think that's pretty much the only way attempting to apply ownership semantics to knowledge or information can end up.
  - balamatom 5mo ago
    Correct. Knowledge consists of, roughly speaking, thoughts. (a "justified true belief" - per https://plato.stanford.edu/entries/knowledge-analysis/ https://plato.stanford.edu/entries/knowledge-analysis/ - is a kind of thought) The "thinking" part of a "thinking being" - that also consists of thoughts. If your knowledges are someone's property, you are someone's property. A society where all knowledge is proprietary, is a society of ubiquitous slavery. Maybe multi-layered, maybe fractional, maybe with a smiley-face drawn on top. Doesn't matter.
- spankalee 5mo ago
  Humans have been known to recite entire parts from plays from memory, live in front of audiences even.
  - leni536 5mo ago
    And they are legally required to license the play to do that, if it's still in copyright.
nkrisc 5mo ago
I find it more ridiculous to equate the act of a human learning with for-profit AI training without recompense to the authors of the training material.
MagicMoonlight 5mo ago
If I “learned” your essay and handed it in, would you be happy with that?
array_key_first 5mo ago
The "learning" isn't learning really. I mean it might be, but if you define learning to be a human endeavor than AI can't learn. It's perfectly reasonable to say it's okay for humans to do something but not okay for a computer program to do the same thing. We don't have to equate AI to humans, that's a choice and usually a bad one.
- aeon_ai 5mo ago
  If one defines 'flying' to be a bird's endeavor, then humans can't fly. Now, if you'll excuse me, I need to catch a metal shuttle that chucks itself through the air on wings.
  - greendestiny 5mo ago
    Sure as a word it can be broad, as a concept in our legal system that should be much more nuanced. The relevant extension of your analogy is should birds be required to obey FAA rules? Or should plane factories be protected as nesting sites?
    - nadermx 5mo ago
      Relevant: https://www.bluewin.ch/en/news/swiss-company-builds-airport-in-bird-sanctuary-2790388.html https://www.bluewin.ch/en/news/swiss-company-builds-airport-...
    - Dylan16807 5mo ago
      It's a relevant extension if you think the ability to learn from a work is a right people have that exempts them from the more general lockdown copyright would impose. If you come at it from the view of copyright being a limited set of control over some areas but not others, then if copyright doesn't block human learning it shouldn't affect anything similar either, unless a specific rule is added to make those situations be handled differently.
- tensor 5mo ago
  It's also perfectly reasonable to say it's ok for a program or machine to do the same thing as a human. This has been the basis for the technological revolution since the dawn of technology.
greendestiny 5mo ago
I think that it's absurd that we've jumped to the conclusion backpropagation in neural networks should be legally treated the same as human learning. I mean I don't think think I could find a better description for following the derivatives of error in reproducing a set of works as creating a "derivative work".
- alok-g 5mo ago
  >> ... we've jumped to the conclusion backpropagation in neural networks should be legally treated the same as human learning. I agree. However, the reverse is also likely true, i.e., it cannot currently be denied that learning in humans is different from learning in artificial neural networks from the point of view of production of works that mix ideas/memes from several works processed/read. Surely, as the article says, copyright law talks exclusively about humans, not machines, not animals.
  - greendestiny 5mo ago
    I understand the article - the point about 'learning' is that if the model and its outputs are a derivative works then the copyright belongs to the human creators of the works it was trained on. Edit*: Or perhaps put more pseudo legally that the created works infringe on the copyrights of the original human creators.
    - alok-g 5mo ago
      The part I agree to is that copyright law calls out humans specifically as the potential owners of copyright. So what you suggest seems to be the only possibility out. Calling out humans could imply that when a human reads a thousand books and then writes something basis the same but which is not a substantial copy of anything explicitly read, that human owns the copyright to the text written. Whereas, if an artificial neural network does the same (hypothetically writing the same text), it would not. The above does not follow from, imply or conclude anything about learning in artificial neural networks and humans being similar or dissimilar.
charonn0 5mo ago
Is "learning" the correct term? Or is it "plagiarism"?
pessimizer 5mo ago
"Learning" for LLMs is just as goofy and propagandistic a metaphor as "stealing" for copyright. I find it predictive of your position that you'll accept one dumb metaphor for something that we didn't need a metaphor for, but not the other. Are you for stealing and against learning? We know exactly what is happening in both cases. We can talk about that, or we can use obfuscating euphemisms that make our preferred position seem obviously true.

I still find the idea that "learning" from code is "stealing" kind of ridiculous.

by tensor 5mo ago

I still find the idea that "learning" from code is "stealing" kind of ridiculous.

estimator7292 5mo ago

Learning, probably not. Copy/pasting at scale, yes

vorticalbox 5mo ago
It is learning though. It’s not just copying the code. Code gets turned into tokens and then it learns the next most likely token. The issue that I see most people talk about it the scale at which is learnt. A human will learn from other people’s code but not from every persons code.
- cogman10 5mo ago
  The issue is that of copyright law WRT to derivative works. Machine transformations on original works does not create a new copyright for the person that directed the machine transformation. That's why you can't pirate a bunch of media by simply adding a red pixel to the righthand corner or by color shifting the video. Copyright law is very clear that if a machine does it, the original copyright on the input is kept. This is why your distributed binaries are still copyrighted, because the machine transformed, very significantly, the source code into binary which maintains the copyright throughout. It would be inconsistent for the courts to suddenly decide that "actually, this specific type of machine transformation is actually innovative." I know this is generally really bad for the AI industry, so they just ignore it until a court tells them they can't anymore. And they might get away with it as I don't have faith that the courts will be consistent.
  - red75prime 5mo ago
    Shredding is a machine transformation. Does it mean that shreds retain original copyright even if the content can't be restored and the provenance can't be traced? Just an example that treating all machine transformations equally with no regard to the specifics doesn't make much sense. And the specifics of autoregressive pretraining is that it is lossy compression. Good luck finding which copyrighted materials have made it into the final weights.
    - cogman10 5mo ago
      > Does it mean that shreds retain original copyright even if the content can't be restored? Yup, it absolutely does. In fact, that's why you are still violating copyright law by using bittorrent even though each of the users is only giving out a small slice or shred of the original content. The US has a granted defense in the case of something like shredding called "Fair Use" but that doesn't mean or imply that a copyright is void simply because of a fair use claim. > And the specifics of autoregressive pretraining is that it is lossy compression. That doesn't matter. Why would it? If I take a FLAC recording and change it to an MP3. The fact that it was a lossy transform doesn't suddenly give me the legal right to distribute the MP3. > Good luck finding which copyrighted materials have made it into the final weights. That's what the NYT v. OpenAI lawsuit is all about. And for earlier models they could, in fact, pull out full NYT articles which proved they made it into the final weights. Further, the NYT is currently in discovery which means OpenAI must open up to the NYT what goes into their weights. A move that, if OpenAI loses, other litigants can also use because there's a real good shot that OpenAI also included their works in the dataset.
      - red75prime 5mo ago
        > Yup, it absolutely does Well, it's not the first time when the law contradicts laws of nature (for the entertainment of the future generations). Bittorent is not a relevant example, because the system is designed to restore the work in its fullness. > in fact, pull out full NYT articles That's when they used their knowledge of the exact text they wanted to "retrieve" to get the text? It wouldn't be so efficient with a random number generator, but it's doable.
        cogman10 5mo ago
        > Bittorent is not a relevant example, because the system is designed to restore the work in its fullness. You can restore shredded documents with enough time and effort. And if you did that and started making photo copies, even if they are incomplete, you will run afoul of copyright law. Bittorrent is a relevant example because it shows that shredding doesn't destroy copyright. Remember, copyright is about the right to copy something. Simply shredding or destroying a thing isn't applicable to copyright. Nor is giving that thing away. What's applicable is when you start to actually copy the thing.
- blks 5mo ago
  A human is not a commercial product. Here we have commercial product that was created by using a lot of various copyrighted and protected IP, without licensing agreements, without paying, without even citing it.
- ell1e 5mo ago
  LLMs seem to be so devoid of intelligence, I think it's arguable if that's learning: https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think... Typically, you would imply a level of understanding when you say learning. LLMs apparently can't do that, by design.
margalabargala 5mo ago
Copy/pasting at scale is how tons of software has been written for a long time, or have we all forgotten the jokes people used to make about StackOverflow?

pydry 5mo ago

If you can set a copyright trap and an LLM reproduces it I think it's pretty clear cut that it's more than just "learning". I have seen LLMs do all sorts of crap which was clearly reproduction of training material. This is also why people are most impressed with how much better it is at reproducing boilerplate rather than, say, imaginative new ideas.

jakeydus 5mo ago
Remember last year (?) when one of the major AIs produced a bit of code that included Jeff Geerling's name in a comment?

lo_zamoyski 5mo ago

If there were the case, then imagine having to give it back!

boh 5mo ago

Yes I guess there's also no such thing as stealing in torrents since the computer "learns" the data and returns it in a transcoded fashion so it's technically not a reproduction. Yes LLMs can reproduce passages from copyrighted works verbatim but that's only because it "learned" it and it's just telling you what it "knows". The mental calisthenics required to justify this stuff must be exhausting.

idle_zealot 5mo ago
> The mental calisthenics required to justify this stuff must be exhausting. It's only exhausting if you think copyright ever reasonably settled the matter of ownership of knowledge and want to morally justify an incoherent set of outcomes that they personally favor. In practice it's primarily been a tool for the powerful party in any dispute to hammer others for disrupting their business model. I think that's pretty much the only way attempting to apply ownership semantics to knowledge or information can end up.
- balamatom 5mo ago
  Correct. Knowledge consists of, roughly speaking, thoughts. (a "justified true belief" - per https://plato.stanford.edu/entries/knowledge-analysis/ https://plato.stanford.edu/entries/knowledge-analysis/ - is a kind of thought) The "thinking" part of a "thinking being" - that also consists of thoughts. If your knowledges are someone's property, you are someone's property. A society where all knowledge is proprietary, is a society of ubiquitous slavery. Maybe multi-layered, maybe fractional, maybe with a smiley-face drawn on top. Doesn't matter.
spankalee 5mo ago
Humans have been known to recite entire parts from plays from memory, live in front of audiences even.
- leni536 5mo ago
  And they are legally required to license the play to do that, if it's still in copyright.

nkrisc 5mo ago

I find it more ridiculous to equate the act of a human learning with for-profit AI training without recompense to the authors of the training material.

MagicMoonlight 5mo ago

If I “learned” your essay and handed it in, would you be happy with that?

array_key_first 5mo ago

The "learning" isn't learning really. I mean it might be, but if you define learning to be a human endeavor than AI can't learn. It's perfectly reasonable to say it's okay for humans to do something but not okay for a computer program to do the same thing. We don't have to equate AI to humans, that's a choice and usually a bad one.

aeon_ai 5mo ago
If one defines 'flying' to be a bird's endeavor, then humans can't fly. Now, if you'll excuse me, I need to catch a metal shuttle that chucks itself through the air on wings.
- greendestiny 5mo ago
  Sure as a word it can be broad, as a concept in our legal system that should be much more nuanced. The relevant extension of your analogy is should birds be required to obey FAA rules? Or should plane factories be protected as nesting sites?
  - nadermx 5mo ago
    Relevant: https://www.bluewin.ch/en/news/swiss-company-builds-airport-in-bird-sanctuary-2790388.html https://www.bluewin.ch/en/news/swiss-company-builds-airport-...
  - Dylan16807 5mo ago
    It's a relevant extension if you think the ability to learn from a work is a right people have that exempts them from the more general lockdown copyright would impose. If you come at it from the view of copyright being a limited set of control over some areas but not others, then if copyright doesn't block human learning it shouldn't affect anything similar either, unless a specific rule is added to make those situations be handled differently.
tensor 5mo ago
It's also perfectly reasonable to say it's ok for a program or machine to do the same thing as a human. This has been the basis for the technological revolution since the dawn of technology.

greendestiny 5mo ago

I think that it's absurd that we've jumped to the conclusion backpropagation in neural networks should be legally treated the same as human learning. I mean I don't think think I could find a better description for following the derivatives of error in reproducing a set of works as creating a "derivative work".

alok-g 5mo ago
>> ... we've jumped to the conclusion backpropagation in neural networks should be legally treated the same as human learning. I agree. However, the reverse is also likely true, i.e., it cannot currently be denied that learning in humans is different from learning in artificial neural networks from the point of view of production of works that mix ideas/memes from several works processed/read. Surely, as the article says, copyright law talks exclusively about humans, not machines, not animals.
- greendestiny 5mo ago
  I understand the article - the point about 'learning' is that if the model and its outputs are a derivative works then the copyright belongs to the human creators of the works it was trained on. Edit*: Or perhaps put more pseudo legally that the created works infringe on the copyrights of the original human creators.
  - alok-g 5mo ago
    The part I agree to is that copyright law calls out humans specifically as the potential owners of copyright. So what you suggest seems to be the only possibility out. Calling out humans could imply that when a human reads a thousand books and then writes something basis the same but which is not a substantial copy of anything explicitly read, that human owns the copyright to the text written. Whereas, if an artificial neural network does the same (hypothetically writing the same text), it would not. The above does not follow from, imply or conclude anything about learning in artificial neural networks and humans being similar or dissimilar.

charonn0 5mo ago

Is "learning" the correct term? Or is it "plagiarism"?

pessimizer 5mo ago

"Learning" for LLMs is just as goofy and propagandistic a metaphor as "stealing" for copyright. I find it predictive of your position that you'll accept one dumb metaphor for something that we didn't need a metaphor for, but not the other. Are you for stealing and against learning? We know exactly what is happening in both cases. We can talk about that, or we can use obfuscating euphemisms that make our preferred position seem obviously true.