3 ms·
Clarifying: the part I'm arguing is "self-evident" is that an LLM is a derivative work of its training data, in the same sense that if you copy the text of a mi
by JoshTriplett 3y ago
Clarifying: the part I'm arguing is "self-evident" is that an LLM is a derivative work of its training data, in the same sense that if you copy the text of a million books into a data file and compress that file reversibly in a way that lets you get most or all of them back out again, the result is clearly a derivative work of those books. That part I made a case for, and it seems like from your last paragraph you agree with that part of the argument.
(By contrast, I wouldn't say it's self-evident that a database of blake3 hashes would be a derivative work (leaving aside that it'd probably be fair use), nor is it self-evident that compiling a million books into a Bloom filter that can recognize any random sentence but not output any random sentence would make the Bloom filter a derivative work. I think the unfiltered LLM being able to output near-verbatim copies of parts of the training set makes that case evident.)
I agree that the second step, of the output of the LLM being a derivative work of the LLM, is less obvious. And I agree that it's going to take case law before people are sure of the answer to that part. I hope the answer is "yes", and I think it'd do substantial harm to Open Source if the answer is a definitive "no".
- mindcrime 3y agoFair enough. I think the distinction between "the model weights" and "the output of the model" was a little blurred when this first started. Sounds like we're closer to "in agreement" than not for the most part.