4 ms·

> Erik Wallace, a computer scientist at the University of California, Berkeley, told me in a phone interview that the lawsuit had “technical inaccuracies” and w

> Erik Wallace, a computer scientist at the University of California, Berkeley, told me in a phone interview that the lawsuit had “technical inaccuracies” and was “stretching the truth a lot.” Wallace pointed out that Stable Diffusion is only a few gigabytes in size—far too small to contain compressed copies of all or even very many of its training images. >In reality, Stable Diffusion works by first converting a user’s prompt into a latent representation: a list of numbers summarizing the contents of the image. Just as you can identify a point on the Earth’s surface based on its latitude and longitude, Stable Diffusion characterizes images based on their “coordinates” in the “picture space.” It then converts this latent representation into an image. Regardless of the lawsuit, this seems like a quite spurious, self-serving argument in and of itself. If the “coordinates” of the image were not equivalent to the image, how would it be possible to fine tune the model on a set of images and recreate their style? This is setting aside the myriad examples where SD regurgitates fairly close copies of individual works. This is a bit like training a model on the works of Shakespeare, but then claiming that the tensors resulting from ingesting the data are not actually Shakespeare’s writing. Oh really, where’d they come from then? Thin air?

GaggiX 4y ago
If trained correctly the model could easily reproduce any arbitrary image if you know its coordinate in the latent space (and you use deterministic sampler), regardless of whether the image is present in the dataset or not, it's also easy to imagine as the coordinates are as big as the samples generated by the diffusion model. You can take an image and use DDIM inversion to find the coordinates.
- threepio 4y ago
  > You can take an image and use DDIM inversion to find the coordinates I'd like to try this. Any suggestions where I can learn how to implement this technique?
- Kim_Bruning 4y ago
  You can use SD models to draw portraits of people (or anime characters, say) based on their description.
justrealist 4y ago
Yes but an alternate view of the copyright is that a person get a degree in English literature and carefully reads all of Shakespeare's writing, and then goes onto write a screenplay which thematically derives from Hamlet. Obviously you get the Shakespeare influence, but the bar on copyright infringement is very high for a reason — we don't want to sue artists for being influenced by another writer.
- bugglebeetle 4y ago
  I don’t really care about the copyright argument as it relates to human learning because I think this is all a Napster-style false flag by capital owners to enclose ALL data on the internet. The intent isn’t to argue about these models from first principles such that we all have IP-restriction free access from them. It’s to make them do enough legally gray things at scale that they get restricted to only a few power players.
  - larry_ralph_ 4y ago
    [flagged]
    - bugglebeetle 4y ago
      I made a reasonable argument: the intent with Stable Diffusion is to do the same thing that many startups have done, i.e. break the law enough that you establish a dominant market position/raise a ton of money, but create enough externalities that the state has to intervene. Then, bribe politicians to legalize your former law breaking and to lock out any of your competitors. This is what myriad companies in the gig economy have done, for example. Instead of reflexively spewing a series of ideological non-sequiturs, perhaps engage with what I actually said.
      - larry_ralph_ 4y ago
        [flagged]
alex_sf 4y ago
> Regardless of the lawsuit, this seems like a quite spurious, self-serving argument in and of itself. If the “coordinates” of the image were not equivalent to the image ... It's simply not. Stable Diffusion 1.5 was trained on over two billion images. Even if we assume the images were only 10kb on average, that would require 20TB of storage space. The pruned 1.5 model is only 4.27GB, or 4,684x smaller. It is fundamentally and provably impossible to achieve that sort of compression ratio on diverse and already compressed images. 'Coordinates of the image' is a layman's explanation of latent space and the training process, it should not be taken literally. > how would it be possible to fine tune the model on a set of images and recreate their style? The fine tune process represents additional training, and to reliably and consistent reproduce a consistent style generally requires overfitting. It is very roughly analagous to an artist using a reference while drawing/painting. > This is setting aside the myriad examples where SD regurgitates fairly close copies of individual works. Some works were overrepresented in the dataset because they are overrepresented in the world. There are substantially more images and variants of the Mona Lisa on the internet than there are Random Twitter Artist's Furry Waifu. This works in much the same way for humans, who are more far more likely to be influenced by the Mona Lisa than Furry Waifu. > This is a bit like training a model on the works of Shakespeare, but then claiming that the tensors resulting from ingesting the data are not actually Shakespeare’s writing. .. They aren't. Shakespeare did not write them. His works can't be reproduced from them. There is, provably, not enough data for it. This is more different than if an individual read and referenced Shakespeare while writing a play, for all that entails. Specifically: whether it's a human or a model generating the content, the content that is generated could be a violation of copyright. You'll note that this is no different to if I copy and paste Mickey Mouse onto a tshirt design, or if I draw him on a piece of paper with a crayon. Should we also be up in arms about copy/paste and crayons? > Oh really, where’d they come from then? Thin air? This hits on a more important point than I think you realize here. Where did inspiration for the human artists come from? Did they spring from a formless void? Or have they seen a lot of images, internalized what they think was important from it, and then generated art from that?
- bugglebeetle 4y ago
  > They aren't. Shakespeare did not write them. His works can't be reproduced from them. There is, provably, not enough data for it. Oh really? Try googling overfitting. Or ask ChatGPT!
unusualmonkey 4y ago
> Regardless of the lawsuit, this seems like a quite spurious, self-serving argument in and of itself. If the “coordinates” of the image were not equivalent to the image, how would it be possible to fine tune the model on a set of images and recreate their style? This is setting aside the myriad examples where SD regurgitates fairly close copies of individual works. This argument confuses an image and the style. The former is copyrightable, the latter is not.
- bugglebeetle 4y ago
  In the US. In France and other countries, they’re not separable.
jacquesm 4y ago
Finding a new form of data compression wouldn't suddenly abolish copyright.
nonbirithm 4y ago
Sounds like a "color of bits" issue. In practice the courts would care about where the latent coordinates originated from, and if it were only possible to derive them from violation of copyright. Same issue with a copy of Titanic ripped from a decaying VHS and compressed into a bad quality AVI uploaded on a torrent tracker. At some point there's still an issue with unauthorized reproduction even if the bits don't match the master copy, because the "color" would still match. And "color" is not a thing that can be represented with data, it's abstract information. https://ansuz.sooke.bc.ca/entry/23 https://ansuz.sooke.bc.ca/entry/23

GaggiX 4y ago

If trained correctly the model could easily reproduce any arbitrary image if you know its coordinate in the latent space (and you use deterministic sampler), regardless of whether the image is present in the dataset or not, it's also easy to imagine as the coordinates are as big as the samples generated by the diffusion model. You can take an image and use DDIM inversion to find the coordinates.

threepio 4y ago
> You can take an image and use DDIM inversion to find the coordinates I'd like to try this. Any suggestions where I can learn how to implement this technique?
Kim_Bruning 4y ago
You can use SD models to draw portraits of people (or anime characters, say) based on their description.

justrealist 4y ago

Yes but an alternate view of the copyright is that a person get a degree in English literature and carefully reads all of Shakespeare's writing, and then goes onto write a screenplay which thematically derives from Hamlet. Obviously you get the Shakespeare influence, but the bar on copyright infringement is very high for a reason — we don't want to sue artists for being influenced by another writer.

bugglebeetle 4y ago
I don’t really care about the copyright argument as it relates to human learning because I think this is all a Napster-style false flag by capital owners to enclose ALL data on the internet. The intent isn’t to argue about these models from first principles such that we all have IP-restriction free access from them. It’s to make them do enough legally gray things at scale that they get restricted to only a few power players.
- larry_ralph_ 4y ago
  [flagged]
  - bugglebeetle 4y ago
    I made a reasonable argument: the intent with Stable Diffusion is to do the same thing that many startups have done, i.e. break the law enough that you establish a dominant market position/raise a ton of money, but create enough externalities that the state has to intervene. Then, bribe politicians to legalize your former law breaking and to lock out any of your competitors. This is what myriad companies in the gig economy have done, for example. Instead of reflexively spewing a series of ideological non-sequiturs, perhaps engage with what I actually said.
    - larry_ralph_ 4y ago
      [flagged]

alex_sf 4y ago

> Regardless of the lawsuit, this seems like a quite spurious, self-serving argument in and of itself. If the “coordinates” of the image were not equivalent to the image ... It's simply not. Stable Diffusion 1.5 was trained on over two billion images. Even if we assume the images were only 10kb on average, that would require 20TB of storage space. The pruned 1.5 model is only 4.27GB, or 4,684x smaller. It is fundamentally and provably impossible to achieve that sort of compression ratio on diverse and already compressed images. 'Coordinates of the image' is a layman's explanation of latent space and the training process, it should not be taken literally. > how would it be possible to fine tune the model on a set of images and recreate their style? The fine tune process represents additional training, and to reliably and consistent reproduce a consistent style generally requires overfitting. It is very roughly analagous to an artist using a reference while drawing/painting. > This is setting aside the myriad examples where SD regurgitates fairly close copies of individual works. Some works were overrepresented in the dataset because they are overrepresented in the world. There are substantially more images and variants of the Mona Lisa on the internet than there are Random Twitter Artist's Furry Waifu. This works in much the same way for humans, who are more far more likely to be influenced by the Mona Lisa than Furry Waifu. > This is a bit like training a model on the works of Shakespeare, but then claiming that the tensors resulting from ingesting the data are not actually Shakespeare’s writing. .. They aren't. Shakespeare did not write them. His works can't be reproduced from them. There is, provably, not enough data for it. This is more different than if an individual read and referenced Shakespeare while writing a play, for all that entails. Specifically: whether it's a human or a model generating the content, the content that is generated could be a violation of copyright. You'll note that this is no different to if I copy and paste Mickey Mouse onto a tshirt design, or if I draw him on a piece of paper with a crayon. Should we also be up in arms about copy/paste and crayons? > Oh really, where’d they come from then? Thin air? This hits on a more important point than I think you realize here. Where did inspiration for the human artists come from? Did they spring from a formless void? Or have they seen a lot of images, internalized what they think was important from it, and then generated art from that?

bugglebeetle 4y ago
> They aren't. Shakespeare did not write them. His works can't be reproduced from them. There is, provably, not enough data for it. Oh really? Try googling overfitting. Or ask ChatGPT!

unusualmonkey 4y ago

> Regardless of the lawsuit, this seems like a quite spurious, self-serving argument in and of itself. If the “coordinates” of the image were not equivalent to the image, how would it be possible to fine tune the model on a set of images and recreate their style? This is setting aside the myriad examples where SD regurgitates fairly close copies of individual works. This argument confuses an image and the style. The former is copyrightable, the latter is not.

bugglebeetle 4y ago
In the US. In France and other countries, they’re not separable.

jacquesm 4y ago

Finding a new form of data compression wouldn't suddenly abolish copyright.

nonbirithm 4y ago

Sounds like a "color of bits" issue. In practice the courts would care about where the latent coordinates originated from, and if it were only possible to derive them from violation of copyright. Same issue with a copy of Titanic ripped from a decaying VHS and compressed into a bad quality AVI uploaded on a torrent tracker. At some point there's still an issue with unauthorized reproduction even if the bits don't match the master copy, because the "color" would still match. And "color" is not a thing that can be represented with data, it's abstract information. https://ansuz.sooke.bc.ca/entry/23 https://ansuz.sooke.bc.ca/entry/23