5 ms·

> Could a large language model trained on data fit under that term? I don’t think so, but the terminology is vague enough that once again I’m not ready to stake

> Could a large language model trained on data fit under that term? I don’t think so, but the terminology is vague enough that once again I’m not ready to stake my reputation on it. This is such an interesting time - when the rules of the game haven't been settled. It was like when Uber was openly flouting established laws with their service. They knew that new laws would be required eventually and so they were willing to push boundaries on ambiguous interpretations until the matter was clearly settled. What surprises me is the stance people take in the face of ambiguity. For example, when I see such vague terms I assume that the reason is so that github has the plausible deniability in using the data for things like model training. I don't mean to say it was written explicitly so that they would be able to use it for that purpose. I mean that the ambiguity, and lack of a desire to clarify it allows them to operate for some time with the plausible deniability. A product manager could make an internal case that a temporary competitive advantage can be realized by interpreting their terms of service in a particular way. Eventually, law is going to catch up. And when it does Github, Microsoft, OpenAI, and all like them will follow the rules. But I, unlike the author, just assume that they are using this window of ambiguity to use the data they have available to gain an advantage. Maybe I am just cynical and the author is more trusting. The truth is probably somewhere in between. They are probably using more than many would feel comfortable with, but probably less than my worst fears.

ChainOfFools 3y ago
And long before Uber, YouTube did it in spectacularly brazen fashion. Back when the site's value was 5% very basic video player, 15% reliable fat pipe' and 80% enormous database of complete "we can't be responsible for policing what our users are uploading" copyright infringment. "Let the rights holders sue us today to take their content down, and beg us tomorrow to keep it up." And it worked, which is how we end up with a hanful of people who would otherwise be competent but unremarkable 9-5 coders at $CORP, are instead through a bit of fortunate timing, in control of fortunes the size of small countries.
- smt88 3y ago
  YouTube doesn't belong on the list. The process you're describing is fully legal under the DMCA and has been the operating mode for every site that allows user submissions.
  - WrongAssumption 3y ago
    I don't see how that negates what they said. If anything it bolsters it. YouTube went live in 1995, the DMCA wasn't passed into law until 1998.
    - nostrademons 3y ago
      YouTube went live in 2005.
      - mrtranscendence 3y ago
        Having used the web in 1995 with a 56k modem, I find the idea of a period-appropriate "YouTube" somewhat amusing. I'm not even sure RealPlayer did video that early.
        nostrademons 3y ago
        RealAudio was released in April 1995. If you listened to RealAudio files (files, because most modems couldn't stream them real-time), the quality was terrible though. I remember the very first MP3 I ever downloaded was a 3:50 song, about 4MB, and took about 2 hours to download. It actually took 2 days of wall-clock time, because my parents kept kicking me off the phone (thank goodness for ZMODEM). And then I found out that my computer (Mac Centris 660AV) was too slow to play it back real-time, so I set about decompressing it. And since it was a 40MB file and I didn't have 40MB space lying around (the whole hard disk was only 230MB), I set about backing up files to ZIP disk until I could. After about a week I finally had my audio file ready, and the experience was magical: near CD quality audio downloaded off the Internet.
  - deleted 3y ago
    [deleted]
wombatpm 3y ago
My fear is that the law will be settled for large corporations in the stupidest manner possible. For example: I took an into to chemical engineering class, bought and read the textbook. Later in my career I taught that class as a graduate student. If I write blog entries about NPSH considerations when sizing pumps, it may seem to be similar to the text book I used oh so many years ago. If an AI uses my blogs for training does the book publisher have a case for infringement? If the AI training owners buy used text books, scan and digitize them, are they not allowed to train their AI on the data? I find all of this hew and cry about training data to just be large corps wanting yet another dip into someone else’s wallet. You published data. There was no license agreement in place, nor do I think you should have one if you also want to claim copyright protection. The fact that someone else may make a buck is just a fact of life and business.
- belorn 3y ago
  In the first example, I would ask what you as a teacher are contributing to your students beyond what exist in the textbook. If the answer is nothing, then what is the difference between a student taking your class compared to listening to a voice synthesizer? I generally suspect that teachers are selling themselves short if they think that their only contribution is to repeat what exist in the textbook. In the second example, how similar are they compared to the textbook? If they are just copied out sections then yes, it most likely is infringement (assuming they are eligible works to begin with). As with any dance around copyright eligibility and fair use exceptions, the devil is in the details. What I hope is that the law won't just create an other situation where only corporations with large departments of lawyers can do things, while any small fry will be shot down by copyright notices, DMCA, blocked accounts and accusation of hacking. For example, if google want to use any published data for training data, then I want to use published videos on youtube to use for the same purpose. If they don't need a license to use other peoples work, then no EULA, DRM, or user agreement should be able to prevent others to do the exact same thing. I suspect however that google only want data to travel in one direction and will do everything in their lawyers power to keep it that way.
  - wombatpm 3y ago
mkl 3y ago
To me the weights of a large language model absolutely seem like aggregate data. The weights are numbers derived from the input data as a whole - in aggregate - and not from individual bits of code.

ChainOfFools 3y ago

And long before Uber, YouTube did it in spectacularly brazen fashion. Back when the site's value was 5% very basic video player, 15% reliable fat pipe' and 80% enormous database of complete "we can't be responsible for policing what our users are uploading" copyright infringment. "Let the rights holders sue us today to take their content down, and beg us tomorrow to keep it up." And it worked, which is how we end up with a hanful of people who would otherwise be competent but unremarkable 9-5 coders at $CORP, are instead through a bit of fortunate timing, in control of fortunes the size of small countries.

smt88 3y ago
YouTube doesn't belong on the list. The process you're describing is fully legal under the DMCA and has been the operating mode for every site that allows user submissions.
- WrongAssumption 3y ago
  I don't see how that negates what they said. If anything it bolsters it. YouTube went live in 1995, the DMCA wasn't passed into law until 1998.
  - nostrademons 3y ago
    YouTube went live in 2005.
    - mrtranscendence 3y ago
      Having used the web in 1995 with a 56k modem, I find the idea of a period-appropriate "YouTube" somewhat amusing. I'm not even sure RealPlayer did video that early.
      - nostrademons 3y ago
        RealAudio was released in April 1995. If you listened to RealAudio files (files, because most modems couldn't stream them real-time), the quality was terrible though. I remember the very first MP3 I ever downloaded was a 3:50 song, about 4MB, and took about 2 hours to download. It actually took 2 days of wall-clock time, because my parents kept kicking me off the phone (thank goodness for ZMODEM). And then I found out that my computer (Mac Centris 660AV) was too slow to play it back real-time, so I set about decompressing it. And since it was a 40MB file and I didn't have 40MB space lying around (the whole hard disk was only 230MB), I set about backing up files to ZIP disk until I could. After about a week I finally had my audio file ready, and the experience was magical: near CD quality audio downloaded off the Internet.
- deleted 3y ago
  [deleted]

wombatpm 3y ago

My fear is that the law will be settled for large corporations in the stupidest manner possible. For example: I took an into to chemical engineering class, bought and read the textbook. Later in my career I taught that class as a graduate student. If I write blog entries about NPSH considerations when sizing pumps, it may seem to be similar to the text book I used oh so many years ago. If an AI uses my blogs for training does the book publisher have a case for infringement? If the AI training owners buy used text books, scan and digitize them, are they not allowed to train their AI on the data? I find all of this hew and cry about training data to just be large corps wanting yet another dip into someone else’s wallet. You published data. There was no license agreement in place, nor do I think you should have one if you also want to claim copyright protection. The fact that someone else may make a buck is just a fact of life and business.

belorn 3y ago
In the first example, I would ask what you as a teacher are contributing to your students beyond what exist in the textbook. If the answer is nothing, then what is the difference between a student taking your class compared to listening to a voice synthesizer? I generally suspect that teachers are selling themselves short if they think that their only contribution is to repeat what exist in the textbook. In the second example, how similar are they compared to the textbook? If they are just copied out sections then yes, it most likely is infringement (assuming they are eligible works to begin with). As with any dance around copyright eligibility and fair use exceptions, the devil is in the details. What I hope is that the law won't just create an other situation where only corporations with large departments of lawyers can do things, while any small fry will be shot down by copyright notices, DMCA, blocked accounts and accusation of hacking. For example, if google want to use any published data for training data, then I want to use published videos on youtube to use for the same purpose. If they don't need a license to use other peoples work, then no EULA, DRM, or user agreement should be able to prevent others to do the exact same thing. I suspect however that google only want data to travel in one direction and will do everything in their lawyers power to keep it that way.
- wombatpm 3y ago

mkl 3y ago

To me the weights of a large language model absolutely seem like aggregate data. The weights are numbers derived from the input data as a whole - in aggregate - and not from individual bits of code.