4 ms·

Seems absurd that somehow the scale being massive makes it better somehow You would think having a massive scale just means it has infringed even more copyrigh

Seems absurd that somehow the scale being massive makes it better somehow You would think having a massive scale just means it has infringed even more copyrights, and therefore should be in even more hot water

TeMPOraL 2y ago
You may or may not agree with it, but that's the only thing that makes it different - scale and non-specificity. Same thing that worked for search engines, for example. My point isn't to argue merits of that case, it's just to point out that OP's joke is like a stereotypical output of an LLM: seems to make sense, but really doesn't.
NewJazz 2y ago
My US history teacher taught me something important. He said that if you are going to steal and don't want to get in trouble, steal a whole lot.
- PontifexMinimus 2y ago
  Copying one person is plagarism. Copying lots of people is research.
  - comfysocks 2y ago
    True, but if you research lots of sources and still emit significant blocks of verbatim text without attribution, it’s still plagiarism. At least that’s how human authors are judged.
    - TeMPOraL 2y ago
      Plagiarism is not illegal, it is merely frowned on, and only in certain fields at that.
      - bayindirh 2y ago
        This is a reductionist take. Maybe it's not illegal per se where you live, but it always have ramifications, and these ramifications affect your future a whole lot.
- psychoslave 2y ago
  Scale might be a factor, but it's not the only one. Your neighbor might not care if you steal a grass stalk in its lawn, and feel powerless if you're the bloody dictator of the country which wastes tremendous amount of resources in socially useless whims thanks to overwhelming taxes. But most people don't want to live in permanent mental distress due to shame of past action or fear of rebellion, I guess.
- throwaway2037 2y ago
  Very interesting post! Can you share more about your teacher's reasoning?
  - SuchAnonMuchWow 2y ago
    It likely comes from the saying similar to this one: "kill a few, you are a murderer. Kill millions, you are a conqueror". More generally, we tend to view number of causalities in war as a large number, and not as the sum of every tragedies that it represent and that we perceive when fewer people die.
omeid2 2y ago
It may not make a lot of sense but it follows the "fair use" doctrine. Which is generally based on the following 4 factors: 1) the purpose and character of use. 2) the nature of the copyrighted material. 3) the *amount* and *substantiality* of the portion taken, and. 4) the effect of the use upon the *potential market*. So in that regard, if you're training a personal assistance GPT, and use some software code to teach your model logic, that is easy to defend as fair use. But the extent of use matters, and if you're training an AI for the sole purpose of regurgitating specific copyrighted material, it is infringement, if it is copyrighted, but in this case, it is not copyright issue, it is contracts and NDAs.
kmeisthax 2y ago
So, the law has this concept of 'de minimus' infringement, where if you take a very small amount - like, way smaller than even a fair use - the courts don't care. If you're taking a handful of word probabilities from every book ever written, then the portion taken from each work is very, very low, so courts aren't likely to care. If you're only training on a handful of works then you're taking more from them, meaning it's not de minimus. For the record, I got this legal theory from Cory Doctorow[0], but I'm skeptical. It's very plausible, but at the same time, we also thought sampling in music was de minimus until the Second Circuit said otherwise. Copyright law is extremely malleable in the presence of moneyed interests, sometimes without Congressional intervention even! [0] who is NOT pro-AI, he just thinks labor law is a better bulwark against it than copyright
- wtallis 2y ago
  If your training process ingests the entire text of the book, and trains with a large context size, you're getting more than just "a handful of word probabilities" from that book.
  - ben_w 2y ago
    If you've trained a 16-bit ten billion parameter model on ten trillion tokens, then the mean training token changes 2/125 of a bit, and a 60k word novel (~75k tokens) contributes 1200 bits. It's up to you if that counts as "a handful" or not.
    - hansworst 2y ago
      I think it’s questionable whether you can actually use this bit count to represent the amount of information from the book. Those 1200 bits represent the way in which this particular book is different from everything else the model has ingested. Similarly, if you read an entire book yourself, your brain will just store the salient bits, not the entire text, unless you have a photographic memory. If we take math or computer science for example: some very important algorithms can be compressed to a few bits of information if you (or a model) have a thorough understanding of the surrounding theory to go with it. Would it not amount to IP infringement if a model regurgitates the relevant information from a patent application, even if it is represented by under a kilobyte of information?
tempodox 2y ago
Almost reminds one of real life: The big thieves get away and have a fan base while the small ones get prosecuted as criminals.
blksv 2y ago
It is the same scale argument that allows you to publish a photo of a procession without written consent from every participant.

TeMPOraL 2y ago

You may or may not agree with it, but that's the only thing that makes it different - scale and non-specificity. Same thing that worked for search engines, for example. My point isn't to argue merits of that case, it's just to point out that OP's joke is like a stereotypical output of an LLM: seems to make sense, but really doesn't.

NewJazz 2y ago

My US history teacher taught me something important. He said that if you are going to steal and don't want to get in trouble, steal a whole lot.

PontifexMinimus 2y ago
Copying one person is plagarism. Copying lots of people is research.
- comfysocks 2y ago
  True, but if you research lots of sources and still emit significant blocks of verbatim text without attribution, it’s still plagiarism. At least that’s how human authors are judged.
  - TeMPOraL 2y ago
    Plagiarism is not illegal, it is merely frowned on, and only in certain fields at that.
    - bayindirh 2y ago
      This is a reductionist take. Maybe it's not illegal per se where you live, but it always have ramifications, and these ramifications affect your future a whole lot.
psychoslave 2y ago
Scale might be a factor, but it's not the only one. Your neighbor might not care if you steal a grass stalk in its lawn, and feel powerless if you're the bloody dictator of the country which wastes tremendous amount of resources in socially useless whims thanks to overwhelming taxes. But most people don't want to live in permanent mental distress due to shame of past action or fear of rebellion, I guess.
throwaway2037 2y ago
Very interesting post! Can you share more about your teacher's reasoning?
- SuchAnonMuchWow 2y ago
  It likely comes from the saying similar to this one: "kill a few, you are a murderer. Kill millions, you are a conqueror". More generally, we tend to view number of causalities in war as a large number, and not as the sum of every tragedies that it represent and that we perceive when fewer people die.

omeid2 2y ago

It may not make a lot of sense but it follows the "fair use" doctrine. Which is generally based on the following 4 factors: 1) the purpose and character of use. 2) the nature of the copyrighted material. 3) the *amount* and *substantiality* of the portion taken, and. 4) the effect of the use upon the *potential market*. So in that regard, if you're training a personal assistance GPT, and use some software code to teach your model logic, that is easy to defend as fair use. But the extent of use matters, and if you're training an AI for the sole purpose of regurgitating specific copyrighted material, it is infringement, if it is copyrighted, but in this case, it is not copyright issue, it is contracts and NDAs.

kmeisthax 2y ago

So, the law has this concept of 'de minimus' infringement, where if you take a very small amount - like, way smaller than even a fair use - the courts don't care. If you're taking a handful of word probabilities from every book ever written, then the portion taken from each work is very, very low, so courts aren't likely to care. If you're only training on a handful of works then you're taking more from them, meaning it's not de minimus. For the record, I got this legal theory from Cory Doctorow[0], but I'm skeptical. It's very plausible, but at the same time, we also thought sampling in music was de minimus until the Second Circuit said otherwise. Copyright law is extremely malleable in the presence of moneyed interests, sometimes without Congressional intervention even! [0] who is NOT pro-AI, he just thinks labor law is a better bulwark against it than copyright

wtallis 2y ago
If your training process ingests the entire text of the book, and trains with a large context size, you're getting more than just "a handful of word probabilities" from that book.
- ben_w 2y ago
  If you've trained a 16-bit ten billion parameter model on ten trillion tokens, then the mean training token changes 2/125 of a bit, and a 60k word novel (~75k tokens) contributes 1200 bits. It's up to you if that counts as "a handful" or not.
  - hansworst 2y ago
    I think it’s questionable whether you can actually use this bit count to represent the amount of information from the book. Those 1200 bits represent the way in which this particular book is different from everything else the model has ingested. Similarly, if you read an entire book yourself, your brain will just store the salient bits, not the entire text, unless you have a photographic memory. If we take math or computer science for example: some very important algorithms can be compressed to a few bits of information if you (or a model) have a thorough understanding of the surrounding theory to go with it. Would it not amount to IP infringement if a model regurgitates the relevant information from a patent application, even if it is represented by under a kilobyte of information?

tempodox 2y ago

Almost reminds one of real life: The big thieves get away and have a fan base while the small ones get prosecuted as criminals.

blksv 2y ago

It is the same scale argument that allows you to publish a photo of a procession without written consent from every participant.