3 ms·

All of them are trained on copyrighted data. Why is it okay for a model to serve up paraphrased books but verbatim copies from the Pirate Bay are illegal? I do

All of them are trained on copyrighted data. Why is it okay for a model to serve up paraphrased books but verbatim copies from the Pirate Bay are illegal? I don’t deny the utility of LLMs. But copyright law was meant to protect authors from this kind of exploitation. Imagine instead of “magical AGI knowledge compression”, instead these LLM providers just did a search over their “borrowed” corpus and then performed a light paraphrasing of it.

kouteiheika 1y ago
> Why is it okay for a model to serve up paraphrased books but verbatim copies from the Pirate Bay are illegal? Because they are not actually memorizing those books (besides few isolated pathological cases due to imperfect training data deduplication), and whatever they spit out is in no way a replacement for the original? Here's some back-of-the-envelope math: Harry Potter and the Philosopher's Stone is around ~460KB of text and equivalent to ~110k Qwen3 tokens, which gives us ~0.24 tokens per byte. Qwen3 models were trained on 36 trillion tokens, so this gives us a dataset of ~137TB. The biggest Qwen3 model has ~235B parameters and at 8-bit (at which you can serve the model essentially loselessly compared to full bf16 weights) takes ~255GB of space, so the model is only 0.18% of its training dataset. And this is the best case, because we took the biggest model, and the actual capacity of a model to memorize is only at most ~4 bit per parameter[1] instead of full ~8 bits we assumed here. For reference, the best loseless compression we can achieve for text is around ~15% of the original size (e.g. Fabrice Bellard's NNCP), which is two orders of magnitude worse. So purely from information theoretic perspective saying that those models memorize the whole datasets on which they were trained is nonsense. They can't do that, because there's just not enough bits to store all of this data. They extract patterns, the same way that I can take the very same ~137TB dataset and build a frequency table of all of the bigrams appearing in it and build a hidden Markov model out of it to generate text. Would that also be "stealing"? And what if I extend my frequency table to trigrams? Where exactly do we draw the line? [1] -- https://arxiv.org/pdf/2505.24832 https://arxiv.org/pdf/2505.24832
- AlexandrB 1y ago
  > whatever they spit out is in no way a replacement for the original? Is this actually true? I think in many cases it is a replacement. Maybe not in the case of a famous fictional work like Harry Potter, but what about non-fiction books or "pulp" fiction? Kind of feels like the bottom rungs of the ladder are being taken out. You either become J.K. Rowling or you starve, there's no room for modest success with AI on the table.
  - otterley 1y ago
    That was the case before AI, too. A small minority of authors are going to be superstars; the rest are still going to have to have day jobs. Publishers largely exist as hedging machines that finance taking risks on new works that have market potential.
terminalshort 1y ago
> Why is it okay for a model to serve up paraphrased books Because that is, and always has been, legal.

kouteiheika 1y ago

> Why is it okay for a model to serve up paraphrased books but verbatim copies from the Pirate Bay are illegal? Because they are not actually memorizing those books (besides few isolated pathological cases due to imperfect training data deduplication), and whatever they spit out is in no way a replacement for the original? Here's some back-of-the-envelope math: Harry Potter and the Philosopher's Stone is around ~460KB of text and equivalent to ~110k Qwen3 tokens, which gives us ~0.24 tokens per byte. Qwen3 models were trained on 36 trillion tokens, so this gives us a dataset of ~137TB. The biggest Qwen3 model has ~235B parameters and at 8-bit (at which you can serve the model essentially loselessly compared to full bf16 weights) takes ~255GB of space, so the model is only 0.18% of its training dataset. And this is the best case, because we took the biggest model, and the actual capacity of a model to memorize is only at most ~4 bit per parameter[1] instead of full ~8 bits we assumed here. For reference, the best loseless compression we can achieve for text is around ~15% of the original size (e.g. Fabrice Bellard's NNCP), which is two orders of magnitude worse. So purely from information theoretic perspective saying that those models memorize the whole datasets on which they were trained is nonsense. They can't do that, because there's just not enough bits to store all of this data. They extract patterns, the same way that I can take the very same ~137TB dataset and build a frequency table of all of the bigrams appearing in it and build a hidden Markov model out of it to generate text. Would that also be "stealing"? And what if I extend my frequency table to trigrams? Where exactly do we draw the line? [1] -- https://arxiv.org/pdf/2505.24832 https://arxiv.org/pdf/2505.24832

AlexandrB 1y ago
> whatever they spit out is in no way a replacement for the original? Is this actually true? I think in many cases it is a replacement. Maybe not in the case of a famous fictional work like Harry Potter, but what about non-fiction books or "pulp" fiction? Kind of feels like the bottom rungs of the ladder are being taken out. You either become J.K. Rowling or you starve, there's no room for modest success with AI on the table.
- otterley 1y ago
  That was the case before AI, too. A small minority of authors are going to be superstars; the rest are still going to have to have day jobs. Publishers largely exist as hedging machines that finance taking risks on new works that have market potential.

terminalshort 1y ago

> Why is it okay for a model to serve up paraphrased books Because that is, and always has been, legal.