3 ms·

There's nothing new about being able to copyright something that's a transformation of another work. And they definitely aren't exclusively trained on public da

There's nothing new about being able to copyright something that's a transformation of another work. And they definitely aren't exclusively trained on public data.

TheDong 1y ago
> There's nothing new about being able to copyright something that's a transformation of another work There is something novel here. Google Books created a huge online index of books, OCRing, compressing them, and transforming them. That was copyright infringement. Just because I download a bunch of copyrighted files and run `tar c | gzip` over them does not mean I have new copyright. Just because I download an image and convert it from png to jpg at 50% quality, throwing away about half the data, does not mean I have created new copyright. AI models are giant lossy compression algorithms. They take text, tokenize it, and turn it into weights, and then inference is a weird form of decompression. See https://bellard.org/ts_zip/ https://bellard.org/ts_zip/ for a logical extension to this. I think this is the reason that the claim of LLM models being unencumbered by copyright is novel. Until now, a human had to do some creative transformation to transform a work, it could not simply be a computer algorithm that changed the format or compressed the input.
- astrange 1y ago
  Google Books is not transformative. It shows you all the same data for the same purpose as they were published for. A better example is Google Image Search. Thumbnails are transformative because they have a different purpose and aren't the same data. An LLM is much more transformative than a thumbnail. It's more lossy than even lossy compression because of the regularization term; I'm pretty sure you can train one that's guaranteed to not retain any of the pretraining text. Of course then it can't answer things like "what's the second line of The Star Spangled Banner".
  - TheDong 1y ago
    Thumbnails are not transformative, they are fair use. They would be copyright infringement, except that a court case ruled them as fair use: https://en.wikipedia.org/wiki/Perfect_10,_Inc._v._Amazon.com,_Inc https://en.wikipedia.org/wiki/Perfect_10,_Inc._v._Amazon.com... . The fact that compression is incredibly lossy does not change the fact that it's copyright infringement. I have a lossy compression algorithm with simply outputs '0' or '1' depending on the parity of bits of the input. If I run that against a camcording of a disney film, the result is a 0 copyrighted by disney, and in fact posting that 0 in this comment would make this comment also illegal so I must disclaim that I did not actually produce that from a camcorded disney film. If I run it against the book 'dracula' the result is a 0 under the public domain. The law does not understand bits, it does not understand compression or lossiness, it understands "humans can creatively transform things, algorithms cannot unless a human imbues creativity into it". It does not matter if your compressed output does not contain the original.
    - astrange 1y ago
      > The court held that framing and hyperlinking of original images for use in an image search engine constituted a fair use of Perfect 10's images because the use was highly transformative ?
  - johanyc 1y ago
    Google Books is transformative. It's a decided case. And it's the same as Google Image, i.e. for search. https://news.ycombinator.com/item?id=45489807 https://news.ycombinator.com/item?id=45489807
    - astrange 1y ago
      Well yeah now it is, otherwise it wouldn't exist. I don't think showing the entire book would be though.
- johanyc 1y ago
  > Google Books created a huge online index of books, OCRing, compressing them, and transforming them. That was copyright infringement. No. It's a decided case. It's transformative and fair use. My understanding why it's transformative is that Google Books mainly offers a search interface for books and it also have measures to make sure only snippets of books are shown.
- dboreham 1y ago
  You're missing something: whether or not it's copyright infringement depends on a) how much money you have and hence bribes you can give and b) whether you can say what you're doing is "to beat China".
  - astrange 1y ago
    Who exactly are you imagining is being bribed here?

There's nothing new about being able to copyright something that's a transformation of another work. And they definitely aren't exclusively trained on public data.

TheDong 1y ago

> There's nothing new about being able to copyright something that's a transformation of another work There is something novel here. Google Books created a huge online index of books, OCRing, compressing them, and transforming them. That was copyright infringement. Just because I download a bunch of copyrighted files and run `tar c | gzip` over them does not mean I have new copyright. Just because I download an image and convert it from png to jpg at 50% quality, throwing away about half the data, does not mean I have created new copyright. AI models are giant lossy compression algorithms. They take text, tokenize it, and turn it into weights, and then inference is a weird form of decompression. See https://bellard.org/ts_zip/ https://bellard.org/ts_zip/ for a logical extension to this. I think this is the reason that the claim of LLM models being unencumbered by copyright is novel. Until now, a human had to do some creative transformation to transform a work, it could not simply be a computer algorithm that changed the format or compressed the input.

astrange 1y ago
Google Books is not transformative. It shows you all the same data for the same purpose as they were published for. A better example is Google Image Search. Thumbnails are transformative because they have a different purpose and aren't the same data. An LLM is much more transformative than a thumbnail. It's more lossy than even lossy compression because of the regularization term; I'm pretty sure you can train one that's guaranteed to not retain any of the pretraining text. Of course then it can't answer things like "what's the second line of The Star Spangled Banner".
- TheDong 1y ago
  Thumbnails are not transformative, they are fair use. They would be copyright infringement, except that a court case ruled them as fair use: https://en.wikipedia.org/wiki/Perfect_10,_Inc._v._Amazon.com,_Inc https://en.wikipedia.org/wiki/Perfect_10,_Inc._v._Amazon.com... . The fact that compression is incredibly lossy does not change the fact that it's copyright infringement. I have a lossy compression algorithm with simply outputs '0' or '1' depending on the parity of bits of the input. If I run that against a camcording of a disney film, the result is a 0 copyrighted by disney, and in fact posting that 0 in this comment would make this comment also illegal so I must disclaim that I did not actually produce that from a camcorded disney film. If I run it against the book 'dracula' the result is a 0 under the public domain. The law does not understand bits, it does not understand compression or lossiness, it understands "humans can creatively transform things, algorithms cannot unless a human imbues creativity into it". It does not matter if your compressed output does not contain the original.
  - astrange 1y ago
    > The court held that framing and hyperlinking of original images for use in an image search engine constituted a fair use of Perfect 10's images because the use was highly transformative ?
- johanyc 1y ago
  Google Books is transformative. It's a decided case. And it's the same as Google Image, i.e. for search. https://news.ycombinator.com/item?id=45489807 https://news.ycombinator.com/item?id=45489807
  - astrange 1y ago
    Well yeah now it is, otherwise it wouldn't exist. I don't think showing the entire book would be though.
johanyc 1y ago
> Google Books created a huge online index of books, OCRing, compressing them, and transforming them. That was copyright infringement. No. It's a decided case. It's transformative and fair use. My understanding why it's transformative is that Google Books mainly offers a search interface for books and it also have measures to make sure only snippets of books are shown.
dboreham 1y ago
You're missing something: whether or not it's copyright infringement depends on a) how much money you have and hence bribes you can give and b) whether you can say what you're doing is "to beat China".
- astrange 1y ago
  Who exactly are you imagining is being bribed here?