3 ms·

Only problem is that LLMs don't "learn" from blog posts in any human sense of the word, they are incapable of symbolic reasoning. LLMs directly incorporate the

Only problem is that LLMs don't "learn" from blog posts in any human sense of the word, they are incapable of symbolic reasoning. LLMs directly incorporate the "learning" material into their models, with superhuman memory capacity and recollection ability, and then remix that source content into their productions. The production might be original, depending on its specific circumstances, but the model and the service itself are without any legal doubt derivative works of the original. They have very little to do with the way humans learn - except maybe if you read a chapter from a book, memorize it, and then proceed to rewrite verbatim with the names of the characters exchanged, a use case of human learning which has long been recognized as, you guessed it, "plagiarism".

dragonwriter 3y ago
> Only problem is that LLMs don’t “learn” from blog posts […] LLMs directly incorporate the learning material into their models The "in-context learning" (regardless of how you feel about the use of that term) that they do from material incorporated into their prompts does not involve incorporating it into their model. > The production might be original, depending on its specific circumstances, but the model and the service itself are without any legal doubt derivative works of the original. Model code is a work, but not derivative of the training data. There’s considerable legal doubt that the models weights themselves are works of authorship at all, which is a prerequisite for being derivative works, so no matter if you are referring to model code or model weights, the statement that model is “without any legal doubt” a derivative work of the training data is false.
- cornholio 3y ago
  You are splitting hairs, there is no doubt that transformers can incorporate and reproduce verbatim training data. Just ask ChatGPT to tell you the lyrics of a famous Madonna song. On that basis alone they are derivative works, regardless of how that data is represented internally.
senko 3y ago
> LLMs directly incorporate the "learning" material into their models. No, that's not how it works. > read a chapter from a book, memorize it, and then proceed to rewrite verbatim with the names of the characters exchanged, That's how a bayesian markov chain would work, not transformers.

dragonwriter 3y ago

> Only problem is that LLMs don’t “learn” from blog posts […] LLMs directly incorporate the learning material into their models The "in-context learning" (regardless of how you feel about the use of that term) that they do from material incorporated into their prompts does not involve incorporating it into their model. > The production might be original, depending on its specific circumstances, but the model and the service itself are without any legal doubt derivative works of the original. Model code is a work, but not derivative of the training data. There’s considerable legal doubt that the models weights themselves are works of authorship at all, which is a prerequisite for being derivative works, so no matter if you are referring to model code or model weights, the statement that model is “without any legal doubt” a derivative work of the training data is false.

cornholio 3y ago
You are splitting hairs, there is no doubt that transformers can incorporate and reproduce verbatim training data. Just ask ChatGPT to tell you the lyrics of a famous Madonna song. On that basis alone they are derivative works, regardless of how that data is represented internally.

senko 3y ago

> LLMs directly incorporate the "learning" material into their models. No, that's not how it works. > read a chapter from a book, memorize it, and then proceed to rewrite verbatim with the names of the characters exchanged, That's how a bayesian markov chain would work, not transformers.