32 ms·

For me that discussion is always hard to grasp. When a human would learn coding autodidacticly by reading source code, and later they would write new code — the

by fxnn 3y ago

For me that discussion is always hard to grasp. When a human would learn coding autodidacticly by reading source code, and later they would write new code — then they could only do so because they read licensed code. No one would ask for the license, right? So why do we care from where LLMs learn?

peoplefromibiza 3y ago
> So why do we care from where LLMs learn? same difference there is between painting your own fake Caravaggio and buying a fake Caravaggio (or selling the one you made). the second one is forgery, the first one is not.
- blackhaz 3y ago
  The way I see it is that with AI you have really painted your own Caravaggio, but instead of an electrochemical circuit of a human brain you've employed a virtual network.
  - peoplefromibiza 3y ago
    > but instead of an electrochemical circuit of a human brain you've employed a virtual network. technically it is still a tool you are using, differently from doing it on your own, with your hands, using your own brain cells, that you trained over the decades, instead of using a virtual electronic brain pre-trained in hours/days by someone else on who knows what.
- joncrocks 3y ago
  And if you look at lots of paintings, and create a new painting which is in a very similar style to an existing painting? Is that a forgery? Have you infringed on the copyright on all the paintings you looked at?
  - jprete 3y ago
    Why do people bring this up? People are not LLMs and the issues are not the same.
    - gaganyaan 3y ago
      Why are the issues not the same? Are you privileging meat over silicon?
      - ddingus 3y ago
        Yes they are. Most people will. They are not the same because an LLM is a construct. It is not a living entity with agency, motive, and all the things the law was intended for. We will see new law as this tech develops. For an analogy, many people call infringement theft and they are wrong to do so. They will focus on the someone getting something without having followed the right process part while ignoring the equally important someone else being denied the use of, or loss of property part. The former is an element in common between theft and infringement. And it is compelling! But, the real meat in theft is all about people losing property! And that is not common at all. This AI thing is similar. The common elements are super compelling. But it just won't be about that in the end. It will be all about the details unique to AI code.
        gaganyaan 3y ago
        Using the word "construct" isn't adding anything to the conversation. If we bioengineer a sentient human, would you feel OK torturing it because it's "just a construct"? If that's unethical to you, how about half meat and half silicon? How much silicon is too much silicon and makes torture OK? > Most people will [privilege meat] "A person is smart. People are dumb, panicky dangerous animals, and you know it". I agree that humans are likely to pass bad laws, because we are mostly just dumb, panicky dangerous animals in the end. That's different than asking an internet commentor why they're being so confident in their opinions though.
        3y ago
- fxnn 3y ago
  Okay if it’s about looking at one painting and fake that. However, if you train your model on billions of paintings and create arbitrary new ones from that, it’s just a statistical analysis on what paintings in general are made of. The importance of the individual painting diminishes at this scale.
sensanaty 3y ago
> So why do we care from where LLMs learn? Because humans aren't computers and the similarities between the two, other than the overuse of the word "learning" in the computer's case, are nonexistant?
- gaganyaan 3y ago
  Are you really asserting that these models aren't learning? What definition of learning are you using?
  - sensanaty 3y ago
    Don't know if they are, and don't really care either and I don't care to anthropomorphize circuitry to the extent that AI proponents tend to, especially. Humans and Computers are 2 wholly separate entities, and there's 0 reason for us to conflate the two. I don't care if another human looks at my code and straight up copies/pastes it, I care very much if an entity backed by a megacorp like Micro$oft does the same, en-masse, and sells it for profit, however.
    - fxnn 3y ago
      Okay, so the scale at which they sale their service is a good argument that this is different from a human learning. However, on the other hand we also have the scale at which they learn, which kind of makes every individual source line of code they learn from pretty unimportant. Learning at this scale is statistical process, and in most cases individual source snippets diminish in the aggregation of millions of others. Or to put it the other way round, the actual value lies in the effort of collecting the samples, training the models, creating the software required for the whole process, putting everything into a good product and selling it. Again, in my mind, the importance of every individual source repo is too small at this scale to care about their license.
      - gaganyaan 3y ago
        It's really not different in scale. Imagine for a moment how much storage space it would take to store the sensory data that any two year old has experienced. That would absolutely dwarf the text-based world the largest of LLMs have experienced.
DrScientist 3y ago
There is the element of the unknown with LLMs etc. There is a legal difference between learning from something and truly making your own version and simply copying. It's vague of course - take plagiarism in a university science essay - the student has no original data and very likely no original thought - but still there is a difference between simply copying a textbook and writing it in your own words. Bottom line - how do we know the output of the LLM isn't a verbatim copy of something with the license stripped off?
red_trumpet 3y ago
Companies don't go around donating their source code to universities either, even if it was only for the purpose of learning.
blitzar 3y ago
> So why do we care from where LLMs learn? Because humans dont put the "Shutterstock" watermark logo on the images they produce.
- defrost 3y ago
  As with all absolutes* exceptions exist: Viagra Boys - In Spite Of Ourselves (with Amy Taylor) I absolutely love that the entirety of the video is unpurchased stock footage with the watermark still on it. This is cinematic gold. https://www.youtube.com/watch?v=WLl1qpDL7YA https://www.youtube.com/watch?v=WLl1qpDL7YA * well, most ...
- hnben 3y ago
  cargo cult programming is real though

by fxnn 3y ago

peoplefromibiza 3y ago

> So why do we care from where LLMs learn? same difference there is between painting your own fake Caravaggio and buying a fake Caravaggio (or selling the one you made). the second one is forgery, the first one is not.

blackhaz 3y ago
The way I see it is that with AI you have really painted your own Caravaggio, but instead of an electrochemical circuit of a human brain you've employed a virtual network.
- peoplefromibiza 3y ago
  > but instead of an electrochemical circuit of a human brain you've employed a virtual network. technically it is still a tool you are using, differently from doing it on your own, with your hands, using your own brain cells, that you trained over the decades, instead of using a virtual electronic brain pre-trained in hours/days by someone else on who knows what.
joncrocks 3y ago
And if you look at lots of paintings, and create a new painting which is in a very similar style to an existing painting? Is that a forgery? Have you infringed on the copyright on all the paintings you looked at?
- jprete 3y ago
  Why do people bring this up? People are not LLMs and the issues are not the same.
  - gaganyaan 3y ago
    Why are the issues not the same? Are you privileging meat over silicon?
    - ddingus 3y ago
      Yes they are. Most people will. They are not the same because an LLM is a construct. It is not a living entity with agency, motive, and all the things the law was intended for. We will see new law as this tech develops. For an analogy, many people call infringement theft and they are wrong to do so. They will focus on the someone getting something without having followed the right process part while ignoring the equally important someone else being denied the use of, or loss of property part. The former is an element in common between theft and infringement. And it is compelling! But, the real meat in theft is all about people losing property! And that is not common at all. This AI thing is similar. The common elements are super compelling. But it just won't be about that in the end. It will be all about the details unique to AI code.
      - gaganyaan 3y ago
        Using the word "construct" isn't adding anything to the conversation. If we bioengineer a sentient human, would you feel OK torturing it because it's "just a construct"? If that's unethical to you, how about half meat and half silicon? How much silicon is too much silicon and makes torture OK? > Most people will [privilege meat] "A person is smart. People are dumb, panicky dangerous animals, and you know it". I agree that humans are likely to pass bad laws, because we are mostly just dumb, panicky dangerous animals in the end. That's different than asking an internet commentor why they're being so confident in their opinions though.
        3y ago
fxnn 3y ago
Okay if it’s about looking at one painting and fake that. However, if you train your model on billions of paintings and create arbitrary new ones from that, it’s just a statistical analysis on what paintings in general are made of. The importance of the individual painting diminishes at this scale.

sensanaty 3y ago

> So why do we care from where LLMs learn? Because humans aren't computers and the similarities between the two, other than the overuse of the word "learning" in the computer's case, are nonexistant?

gaganyaan 3y ago
Are you really asserting that these models aren't learning? What definition of learning are you using?
- sensanaty 3y ago
  Don't know if they are, and don't really care either and I don't care to anthropomorphize circuitry to the extent that AI proponents tend to, especially. Humans and Computers are 2 wholly separate entities, and there's 0 reason for us to conflate the two. I don't care if another human looks at my code and straight up copies/pastes it, I care very much if an entity backed by a megacorp like Micro$oft does the same, en-masse, and sells it for profit, however.
  - fxnn 3y ago
    Okay, so the scale at which they sale their service is a good argument that this is different from a human learning. However, on the other hand we also have the scale at which they learn, which kind of makes every individual source line of code they learn from pretty unimportant. Learning at this scale is statistical process, and in most cases individual source snippets diminish in the aggregation of millions of others. Or to put it the other way round, the actual value lies in the effort of collecting the samples, training the models, creating the software required for the whole process, putting everything into a good product and selling it. Again, in my mind, the importance of every individual source repo is too small at this scale to care about their license.
    - gaganyaan 3y ago
      It's really not different in scale. Imagine for a moment how much storage space it would take to store the sensory data that any two year old has experienced. That would absolutely dwarf the text-based world the largest of LLMs have experienced.

DrScientist 3y ago

There is the element of the unknown with LLMs etc. There is a legal difference between learning from something and truly making your own version and simply copying. It's vague of course - take plagiarism in a university science essay - the student has no original data and very likely no original thought - but still there is a difference between simply copying a textbook and writing it in your own words. Bottom line - how do we know the output of the LLM isn't a verbatim copy of something with the license stripped off?

red_trumpet 3y ago

Companies don't go around donating their source code to universities either, even if it was only for the purpose of learning.

blitzar 3y ago

> So why do we care from where LLMs learn? Because humans dont put the "Shutterstock" watermark logo on the images they produce.

defrost 3y ago
As with all absolutes* exceptions exist: Viagra Boys - In Spite Of Ourselves (with Amy Taylor) I absolutely love that the entirety of the video is unpurchased stock footage with the watermark still on it. This is cinematic gold. https://www.youtube.com/watch?v=WLl1qpDL7YA https://www.youtube.com/watch?v=WLl1qpDL7YA * well, most ...
hnben 3y ago
cargo cult programming is real though