3 ms·

The legal cases don't "mean anything" because AI training is /legal/, not because courts are "corrupt". If anything is transformative, an AI that doesn't memori

The legal cases don't "mean anything" because AI training is /legal/, not because courts are "corrupt". If anything is transformative, an AI that doesn't memorize its input is.

nextaccountic 3y ago
Lossy compression of a 1MB original image into a 20kb compressed image doesn't make copyright go away But that's essentially what LLMs are doing, lossy compression of the entire web
bayindirh 3y ago
Yet gleefully emits its training data when one asks the right questions. It can be code, prose or images. Yeah, doesn't remember. Mhm... Oh, it just can't remember the license terms of the code it "reads", so it can't comply with these licenses or help people to comply with these licenses. Convenient.
ben_w 3y ago
> If anything is transformative, an AI that doesn't memorize its input is. I suspect the answer to the question "is it, though?" is one for the lawyers and lawmakers rather than for the software developers, and it may well vary wildly by jurisdiction.
- __loam 3y ago
  Fair use specifically has a clause about disrupting the market for the original work lol. Being transformative isn't the only aspect of fair use, and even if training is legal, you're still a douche for training on art without permission.
nologic01 3y ago
It doesnt memorize anything. It just needs gazillion parameters that approach the size of the training set to finesse its conversational accent.
- astrange 3y ago
  LLama2 has a 5TB training set.
  - nologic01 3y ago
    So? You just support my point. That is a factor of 100-1000 versus model parameter count, assuming that the training set has no redundancy whatsoever. Hence more likely a factor of 10-100. People dont want to acknowledge that the LLM structure reflects rather closely what it is being trained on, but the incredibly large number of parameters suggests it is closer to a photographic fit than a true abstraction. larger models being more likely to memorize training data (Carlini et al., 2021, 2022) The fact that the information gets mangled and somewhat compressed doesnt change this close relationship.

The legal cases don't "mean anything" because AI training is /legal/, not because courts are "corrupt". If anything is transformative, an AI that doesn't memorize its input is.

nextaccountic 3y ago

Lossy compression of a 1MB original image into a 20kb compressed image doesn't make copyright go away But that's essentially what LLMs are doing, lossy compression of the entire web

bayindirh 3y ago

Yet gleefully emits its training data when one asks the right questions. It can be code, prose or images. Yeah, doesn't remember. Mhm... Oh, it just can't remember the license terms of the code it "reads", so it can't comply with these licenses or help people to comply with these licenses. Convenient.

ben_w 3y ago

> If anything is transformative, an AI that doesn't memorize its input is. I suspect the answer to the question "is it, though?" is one for the lawyers and lawmakers rather than for the software developers, and it may well vary wildly by jurisdiction.

__loam 3y ago
Fair use specifically has a clause about disrupting the market for the original work lol. Being transformative isn't the only aspect of fair use, and even if training is legal, you're still a douche for training on art without permission.

nologic01 3y ago

It doesnt memorize anything. It just needs gazillion parameters that approach the size of the training set to finesse its conversational accent.

astrange 3y ago
LLama2 has a 5TB training set.
- nologic01 3y ago
  So? You just support my point. That is a factor of 100-1000 versus model parameter count, assuming that the training set has no redundancy whatsoever. Hence more likely a factor of 10-100. People dont want to acknowledge that the LLM structure reflects rather closely what it is being trained on, but the incredibly large number of parameters suggests it is closer to a photographic fit than a true abstraction. larger models being more likely to memorize training data (Carlini et al., 2021, 2022) The fact that the information gets mangled and somewhat compressed doesnt change this close relationship.