3 ms·

> Weights aren't a compressed representation of the data it was trained on because 99.99% of it can't be recovered verbatim. Cool. So a low quality JPEG isn't

> Weights aren't a compressed representation of the data it was trained on because 99.99% of it can't be recovered verbatim. Cool. So a low quality JPEG isn't a compressed representation?

zirgs 4y ago
Good luck suing me if I resized your photo to 1x1 resolution and posted it on my website. That's how little info is extracted from each image on average.
- BaseballPhysics 4y ago
  Cool, can I also make up statistics? How about we use actual examples: Getty is suing Stability because the model has captured so much information about its training set that it reproduces Getty watermarks in its results: https://arstechnica.com/tech-policy/2023/02/getty-sues-stability-ai-for-copying-12m-photos-and-imitating-famous-watermark/ https://arstechnica.com/tech-policy/2023/02/getty-sues-stabi... Github's Co-pilot can be coaxed into reproducing entire chunks of copyrighted code: https://twitter.com/docsparse/status/1581461734665367554 https://twitter.com/docsparse/status/1581461734665367554 But sure, go ahead, keep trying to convince me these models (particularly once they get large enough) aren't compressed representations of information represented in the underlying training data.
michaelmrose 4y ago
The law isn't settled on this but whatever the final answer is it will clearly not be resolved by relying on poor analogy. A model is NOTHING like a jpeg. It's also little like a human being learning. We are going to need to be adults and pass actual legislation speaking to this.
- BaseballPhysics 4y ago
  You completely missed the point of my comment. The commenter was claiming that unless you can recover data "verbatim" an encoding cannot be considered compression. My example was to illustrate the silliness of that point, not to draw a straight line comparison between model weights and a lossy image compression format. As for legislation, I completely agree with you. There's real danger that, absent that, you'll just end up with legislation from the bench where judges attempt to shoehorn these issues into existing legal frameworks, and I suspect no one will be satisfied with the outcome.

> Weights aren't a compressed representation of the data it was trained on because 99.99% of it can't be recovered verbatim. Cool. So a low quality JPEG isn't a compressed representation?

zirgs 4y ago

Good luck suing me if I resized your photo to 1x1 resolution and posted it on my website. That's how little info is extracted from each image on average.

BaseballPhysics 4y ago
Cool, can I also make up statistics? How about we use actual examples: Getty is suing Stability because the model has captured so much information about its training set that it reproduces Getty watermarks in its results: https://arstechnica.com/tech-policy/2023/02/getty-sues-stability-ai-for-copying-12m-photos-and-imitating-famous-watermark/ https://arstechnica.com/tech-policy/2023/02/getty-sues-stabi... Github's Co-pilot can be coaxed into reproducing entire chunks of copyrighted code: https://twitter.com/docsparse/status/1581461734665367554 https://twitter.com/docsparse/status/1581461734665367554 But sure, go ahead, keep trying to convince me these models (particularly once they get large enough) aren't compressed representations of information represented in the underlying training data.

michaelmrose 4y ago

The law isn't settled on this but whatever the final answer is it will clearly not be resolved by relying on poor analogy. A model is NOTHING like a jpeg. It's also little like a human being learning. We are going to need to be adults and pass actual legislation speaking to this.

BaseballPhysics 4y ago
You completely missed the point of my comment. The commenter was claiming that unless you can recover data "verbatim" an encoding cannot be considered compression. My example was to illustrate the silliness of that point, not to draw a straight line comparison between model weights and a lossy image compression format. As for legislation, I completely agree with you. There's real danger that, absent that, you'll just end up with legislation from the bench where judges attempt to shoehorn these issues into existing legal frameworks, and I suspect no one will be satisfied with the outcome.