6 ms·

> Many clearly think it's a photocopier. As if StableDiffusion is hiding copies of all these works inside. In my latest series of tweets [1] I tried to argue t

> Many clearly think it's a photocopier. As if StableDiffusion is hiding copies of all these works inside. In my latest series of tweets [1] I tried to argue that this is not the case in a less technical way, not sure if I was successful though. For anyone not fluent in Korean, my argument was as follows: Stable Diffusion was trained with a data set composed of about 2.32 billion text-image pairs. The model itself by comparison weighs about 4 GB. If the model somehow stored those images (completely disregarding texts) in the compressed form, it would be less than 2 bytes (EDITED) per each image. Whoever downloaded an image from the internet can see this is absurd. In reality 4 GB of data is the juice extracted from the data set, and if you could actually recover an original image from it then the juice must have had a recognizable residue, which would be pretty undesirable. [1] https://twitter.com/senokay/status/1565543214068600832 https://twitter.com/senokay/status/1565543214068600832

wccrawford 4y ago
20 bytes? Wouldn't that be about 2 bytes per image?
- lifthrasiir 4y ago
  Ah, yes, a sloppy calculation in my part. I probably wanted to say something like 20 bits in the original tweet.
minimaxir 4y ago
Textual inversion also takes an entire concept (including context, style, and 3D attributes) and fits it into 768 float32s / 4KB.
- lifthrasiir 4y ago
  That's a different topic, as you need the entire model to "decompress" that 4 KB data. I'm talking about the model itself.
Animats 4y ago
Another discovery that something thought to be uniquely human really isn't that hard.
- catach 4y ago
  It seems to me that the human effort spent in creating the original images in an AI's data set is an important element to consider when thinking of the "difficulty" here.
  - lifthrasiir 4y ago
    Yes, that 4 GB of data can't be really independently generated, but the fact that a large chunk of that effort can be compressed into 4 GB is still amazing.
    - CamperBob2 4y ago
      If your life's work can be compressed down to 2 bytes, it wasn't that hard. OTOH, if the 2 bytes is an index into a larger corpus of collected works, the question becomes more interesting. OTOH, if the corpus is an ensemble of other 2-byte objects...
  - causality0 4y ago
    Certainly, but those human artists were also trained on a data set their whole lives. Raise a baby with no training data set and they might produce a handprint on a wall or a rectangle of facepaint. There's a reason indigenous tribesmen don't spontaneously invent hyperrealism.
    - catach 4y ago
      Absolutely. Though I think that speaks to one of the ways we tend to discount the difficulty of developing human artistry, rather than a defense of AI gen being "not hard".
    - bayindirh 4y ago
      However, there's a major misconception about training of humans and AI. Image generating AIs are trained with massive amounts of images and text, however image generating humans train with much broader spectrum of experience. Also, feeding a model tons of images created by humans (directly or by proxy) and claiming that AI is generating something completely new is a bit naive IMHO. Humans mix a much more broader and deeper experience pool to create things without prompts. An AI model blurts out something derived from a corpus of images and text created by humans, that's all. The technology is impressive for sure, and it marks a new era in terms of possibilities, but it doesn't take my breath away, sorry.
paranoidrobot 4y ago
It might not be storing all of the images, but it's clearly storing copies of some of the source images. A few weeks before OpenAI made DALL-E 2 open, I went through a series of prompts using well known artists as a basis. Vermeer was one of them. It generated some pretty amazing works that were 'inspired' but not direct copies of Vermeer's works. I started feeding the same prompts into StableDiffiusion (the HuggingFace online demo) and it output direct copies of Vermeer's works with minimal modifications.
- lifthrasiir 4y ago
  Do you have the exact prompt that essentially generated Vemeer's works? Omitted in the parent, but I also noted that while it is hard to reconcile the idea of human-directed transformative use with ML models, it might be actually easier to verify the models' plagarism---which is not equivalent to copyright infringement but can be a problem by its own---once you have a right prompt as they are decidable.
  - paranoidrobot 4y ago
    "the girl with a pearl earring by Johannes Vermeer" and variations on that, will spit out the named painting. Two examples: https://imgur.com/a/KAxmZCl https://imgur.com/a/KAxmZCl The original: https://en.wikipedia.org/wiki/Girl_with_a_Pearl_Earring#/media/File:1665_Girl_with_a_Pearl_Earring.jpg https://en.wikipedia.org/wiki/Girl_with_a_Pearl_Earring#/med... It's reproducing the folds in the headband, riffles in the clothing, etc. It's not just a "similar" reproduced image, it's clearly some minor variation on the original.
  - comex 4y ago
    I just typed in 'girl with a pearl earring vermeer' and got this: https://drop.qoid.us/vermeer.png https://drop.qoid.us/vermeer.png And, for a slightly different scenario, here's what I got for 'pikachu': https://drop.qoid.us/pikachu.png https://drop.qoid.us/pikachu.png But I think this sort of thing is the exception rather than the rule (similar to Copilot regurgitating the text of software licenses). Girl with a Pearl Earring is so famous that there are countless copies of it on the internet. I don't know what kind of deduplication process was used for Stable Diffusion, but if you consider multiple photos of the same painting, plus images distorted by compression, or cropped, or embedded in other images, plus attempts to redraw it or even parody it, I can easily imagine that the training set contained thousands of images or more that are essentially the same work. And so the model memorized it, albeit at low resolution. Similarly, it's seen countless different images of Pikachu. However, this seems to only apply to the very peak of popularity. I tested some other Pokemon: it does a decent job with Charizard, but with Bulbasaur and Squirtle and Jigglypuff its output only somewhat resembles the character design, and for less popular Pokemon there's no resemblance at all.
nwienert 4y ago
Not a good argument, because this type of AI is basically exactly a compression algorithm - it’s the most efficient way we know to compress 2 billion image/text pairs down such that it can recreate the average of all the inputs retaining as much layered knowledge as it can. It’s a very very fancy compression algorithm, but it still is one.
- lifthrasiir 4y ago
  There is an important distinction between lossy compression and lossless compression. My information-theoretic argument shows that models couldn't have been losslessly compressed, and the juice analogy implies (but does not show) that there is no obvious way to "decompress" the original data set. I intended that, if both arguments are true, those models can't be a photocopier because they don't contain copiable works at all.
  - nwienert 4y ago
    The law as far as I know is concerned with derivation not exact copy. It’s easy for these models to produce very close images to the source material with the right short input.
dtn 4y ago
The model size:training image ratio seems more like an implementation detail more than anything. The fact that people are writing prompts with named artists, and phrases such as "Trending on Artstation"- yeah, I'm not sure if you can just handwave that away.
- astrange 4y ago
  In some ways it’s just a coincidence that those prompts do something. You can use names of artists that don’t exist and those work just as well, once you’ve figured out what they mean. Also, even if the model didn’t come with knowledge of an existing artist someone could fine-tune it in, and it’s possible the model can learn about your art style without seeing any of your images anyway.
  - AJ007 4y ago
    This is a very important point. Others have alluded to this, but I'll make a larger claim: human artists actually are doing very little that is original. The styles, composition, and subjects are all derivative works. That means their "essence" can be recreated without them. Another fascinating consideration, if artists names were excised from future models, either by law or by choice, the "mirror" artists (fake) could be the ones which become famous and the humans forgotten. AI could become more generalized or there may be millions of models, doing specific things, strung together like with APIs. Either way, if an artist or trademark owner chooses some kind of explicit blocking, from the input side, it is possible that it would be the equivalent of your keyword censored from Google's index, Google Maps, Amazon, and so on. Disney? I don't recognize that word.
- alextheparrot 4y ago
  “Trending on Artstation” is a genre curated just like Bauhaus or Romanticism. Just the wider populace votes instead of old noble families and wealthy bankers. I don’t see the moral significance of the former genres versus the latter
beezlewax 4y ago
I've used the words "stock photography" in an image and saw the shutterstock logo across the image in the output. Not sure how that happens but try it out.
saving_throw 4y ago
It doesn't compress each image separately. Rather, it compresses its entire training set into one dense region of cartesian space. Each pixel of each training image is in there, scurnched up next to every other pixel of every other training image. It's like sardines in a can, except in thousands (?) of dimensions. You have no hope of calculating how much space it needs for that just by dividing the size of the model by the dimensions of images in pixels, like you did. You can see that it has memorised all its training material very clearly in its ability to perform style transfer. You can ask it to geneate an image of Pikachu in the style of Vermeer. It can do that because it has memorised images of Pikachu and images by Vermeer in its weird compressed form and it can combine them by finding a point on a gradient between their representation in its memory. If it didn't have memorised images of Pikachu, or images by Vermeer, how do you guess it could generate images of Pikachu or in the style of Vermeer? It's not magick.
- saving_throw 4y ago
  > ... sardines in a can ... Better analogy: suppose you take 10 pieces of A4 paper and scrunch them up into 10 separate balls. Now suppose you scrunch them all up together into one ball. One manifold, taking ten times less space. In the same way, you don't need to store each pixel of each image separately, or in its own sub-space. Yes? It's really good compression, that's all.

wccrawford 4y ago

20 bytes? Wouldn't that be about 2 bytes per image?

lifthrasiir 4y ago
Ah, yes, a sloppy calculation in my part. I probably wanted to say something like 20 bits in the original tweet.

minimaxir 4y ago

Textual inversion also takes an entire concept (including context, style, and 3D attributes) and fits it into 768 float32s / 4KB.

lifthrasiir 4y ago
That's a different topic, as you need the entire model to "decompress" that 4 KB data. I'm talking about the model itself.

Animats 4y ago

Another discovery that something thought to be uniquely human really isn't that hard.

catach 4y ago
It seems to me that the human effort spent in creating the original images in an AI's data set is an important element to consider when thinking of the "difficulty" here.
- lifthrasiir 4y ago
  Yes, that 4 GB of data can't be really independently generated, but the fact that a large chunk of that effort can be compressed into 4 GB is still amazing.
  - CamperBob2 4y ago
    If your life's work can be compressed down to 2 bytes, it wasn't that hard. OTOH, if the 2 bytes is an index into a larger corpus of collected works, the question becomes more interesting. OTOH, if the corpus is an ensemble of other 2-byte objects...
- causality0 4y ago
  Certainly, but those human artists were also trained on a data set their whole lives. Raise a baby with no training data set and they might produce a handprint on a wall or a rectangle of facepaint. There's a reason indigenous tribesmen don't spontaneously invent hyperrealism.
  - catach 4y ago
    Absolutely. Though I think that speaks to one of the ways we tend to discount the difficulty of developing human artistry, rather than a defense of AI gen being "not hard".
  - bayindirh 4y ago
    However, there's a major misconception about training of humans and AI. Image generating AIs are trained with massive amounts of images and text, however image generating humans train with much broader spectrum of experience. Also, feeding a model tons of images created by humans (directly or by proxy) and claiming that AI is generating something completely new is a bit naive IMHO. Humans mix a much more broader and deeper experience pool to create things without prompts. An AI model blurts out something derived from a corpus of images and text created by humans, that's all. The technology is impressive for sure, and it marks a new era in terms of possibilities, but it doesn't take my breath away, sorry.

paranoidrobot 4y ago

It might not be storing all of the images, but it's clearly storing copies of some of the source images. A few weeks before OpenAI made DALL-E 2 open, I went through a series of prompts using well known artists as a basis. Vermeer was one of them. It generated some pretty amazing works that were 'inspired' but not direct copies of Vermeer's works. I started feeding the same prompts into StableDiffiusion (the HuggingFace online demo) and it output direct copies of Vermeer's works with minimal modifications.

lifthrasiir 4y ago
Do you have the exact prompt that essentially generated Vemeer's works? Omitted in the parent, but I also noted that while it is hard to reconcile the idea of human-directed transformative use with ML models, it might be actually easier to verify the models' plagarism---which is not equivalent to copyright infringement but can be a problem by its own---once you have a right prompt as they are decidable.
- paranoidrobot 4y ago
  "the girl with a pearl earring by Johannes Vermeer" and variations on that, will spit out the named painting. Two examples: https://imgur.com/a/KAxmZCl https://imgur.com/a/KAxmZCl The original: https://en.wikipedia.org/wiki/Girl_with_a_Pearl_Earring#/media/File:1665_Girl_with_a_Pearl_Earring.jpg https://en.wikipedia.org/wiki/Girl_with_a_Pearl_Earring#/med... It's reproducing the folds in the headband, riffles in the clothing, etc. It's not just a "similar" reproduced image, it's clearly some minor variation on the original.
- comex 4y ago
  I just typed in 'girl with a pearl earring vermeer' and got this: https://drop.qoid.us/vermeer.png https://drop.qoid.us/vermeer.png And, for a slightly different scenario, here's what I got for 'pikachu': https://drop.qoid.us/pikachu.png https://drop.qoid.us/pikachu.png But I think this sort of thing is the exception rather than the rule (similar to Copilot regurgitating the text of software licenses). Girl with a Pearl Earring is so famous that there are countless copies of it on the internet. I don't know what kind of deduplication process was used for Stable Diffusion, but if you consider multiple photos of the same painting, plus images distorted by compression, or cropped, or embedded in other images, plus attempts to redraw it or even parody it, I can easily imagine that the training set contained thousands of images or more that are essentially the same work. And so the model memorized it, albeit at low resolution. Similarly, it's seen countless different images of Pikachu. However, this seems to only apply to the very peak of popularity. I tested some other Pokemon: it does a decent job with Charizard, but with Bulbasaur and Squirtle and Jigglypuff its output only somewhat resembles the character design, and for less popular Pokemon there's no resemblance at all.

nwienert 4y ago

Not a good argument, because this type of AI is basically exactly a compression algorithm - it’s the most efficient way we know to compress 2 billion image/text pairs down such that it can recreate the average of all the inputs retaining as much layered knowledge as it can. It’s a very very fancy compression algorithm, but it still is one.

lifthrasiir 4y ago
There is an important distinction between lossy compression and lossless compression. My information-theoretic argument shows that models couldn't have been losslessly compressed, and the juice analogy implies (but does not show) that there is no obvious way to "decompress" the original data set. I intended that, if both arguments are true, those models can't be a photocopier because they don't contain copiable works at all.
- nwienert 4y ago
  The law as far as I know is concerned with derivation not exact copy. It’s easy for these models to produce very close images to the source material with the right short input.

dtn 4y ago

The model size:training image ratio seems more like an implementation detail more than anything. The fact that people are writing prompts with named artists, and phrases such as "Trending on Artstation"- yeah, I'm not sure if you can just handwave that away.

astrange 4y ago
In some ways it’s just a coincidence that those prompts do something. You can use names of artists that don’t exist and those work just as well, once you’ve figured out what they mean. Also, even if the model didn’t come with knowledge of an existing artist someone could fine-tune it in, and it’s possible the model can learn about your art style without seeing any of your images anyway.
- AJ007 4y ago
  This is a very important point. Others have alluded to this, but I'll make a larger claim: human artists actually are doing very little that is original. The styles, composition, and subjects are all derivative works. That means their "essence" can be recreated without them. Another fascinating consideration, if artists names were excised from future models, either by law or by choice, the "mirror" artists (fake) could be the ones which become famous and the humans forgotten. AI could become more generalized or there may be millions of models, doing specific things, strung together like with APIs. Either way, if an artist or trademark owner chooses some kind of explicit blocking, from the input side, it is possible that it would be the equivalent of your keyword censored from Google's index, Google Maps, Amazon, and so on. Disney? I don't recognize that word.
alextheparrot 4y ago
“Trending on Artstation” is a genre curated just like Bauhaus or Romanticism. Just the wider populace votes instead of old noble families and wealthy bankers. I don’t see the moral significance of the former genres versus the latter

beezlewax 4y ago

I've used the words "stock photography" in an image and saw the shutterstock logo across the image in the output. Not sure how that happens but try it out.

saving_throw 4y ago

It doesn't compress each image separately. Rather, it compresses its entire training set into one dense region of cartesian space. Each pixel of each training image is in there, scurnched up next to every other pixel of every other training image. It's like sardines in a can, except in thousands (?) of dimensions. You have no hope of calculating how much space it needs for that just by dividing the size of the model by the dimensions of images in pixels, like you did. You can see that it has memorised all its training material very clearly in its ability to perform style transfer. You can ask it to geneate an image of Pikachu in the style of Vermeer. It can do that because it has memorised images of Pikachu and images by Vermeer in its weird compressed form and it can combine them by finding a point on a gradient between their representation in its memory. If it didn't have memorised images of Pikachu, or images by Vermeer, how do you guess it could generate images of Pikachu or in the style of Vermeer? It's not magick.

saving_throw 4y ago
> ... sardines in a can ... Better analogy: suppose you take 10 pieces of A4 paper and scrunch them up into 10 separate balls. Now suppose you scrunch them all up together into one ball. One manifold, taking ten times less space. In the same way, you don't need to store each pixel of each image separately, or in its own sub-space. Yes? It's really good compression, that's all.