3 ms·

Copyright infringement is about making a copy. It is not making a copy if you read a book. It is not making a copy if you watch TV. It is not making a copy if y

by evdubs 3y ago

Copyright infringement is about making a copy. It is not making a copy if you read a book. It is not making a copy if you watch TV. It is not making a copy if you listen to the radio. If Denys makes a video/audio file of himself quoting a large segment from "Friends", that's copyright infringement. When machines train on a corpus of data, that corpus data is a copy and is almost certainly infringing on copyright (depending on how they acquired it). When machines use their training model and reproduce verbatim copyrighted works that they've been trained with, that copy is infringing (depending on licenses they acquired to reproduce copyrighted data).

PaulHoule 3y ago
When I read a book I read a "copy" of that book, which was typically reproduced on paper under contract with the copyright owner. The model is not capable of reproducing the input verbatim (not enough synapses to do that and it wouldn't generalize properly if there were) although the training process does involve producing a cleaned up text which is typically retained. (The point of building your language skills from watching "Friends" or an LLM developing linguistic/world knowledge competence or the simulation thereof from reading text is not that the text can be regurgitated, but that you or the the model develops a generalized capacity.) (... if LLMs were able to remember everything they read, why do we always catch them confabulating?) It would be possible in principle to point a camera at the book, OCR it in real time, and feed that into the training process. Of course this would be slow and wouldn't benefit from the cleanup process that goes into making a training set. Note that somebody could stick a fine grid of wires into my V1 visual cortex and extract something that looks very much like an image of what I am focusing on at the moment. Thus, for just a few milliseconds an image "copy" of a text I am reading exists in my brain.

by evdubs 3y ago

PaulHoule 3y ago

When I read a book I read a "copy" of that book, which was typically reproduced on paper under contract with the copyright owner. The model is not capable of reproducing the input verbatim (not enough synapses to do that and it wouldn't generalize properly if there were) although the training process does involve producing a cleaned up text which is typically retained. (The point of building your language skills from watching "Friends" or an LLM developing linguistic/world knowledge competence or the simulation thereof from reading text is not that the text can be regurgitated, but that you or the the model develops a generalized capacity.) (... if LLMs were able to remember everything they read, why do we always catch them confabulating?) It would be possible in principle to point a camera at the book, OCR it in real time, and feed that into the training process. Of course this would be slow and wouldn't benefit from the cleanup process that goes into making a training set. Note that somebody could stick a fine grid of wires into my V1 visual cortex and extract something that looks very much like an image of what I am focusing on at the moment. Thus, for just a few milliseconds an image "copy" of a text I am reading exists in my brain.