4 ms·

This is where it gets interesting. The model is just 4.5gb. That’s too small to contain the actual trillion images it was trained on. So in all likelihood, it d

This is where it gets interesting. The model is just 4.5gb. That’s too small to contain the actual trillion images it was trained on. So in all likelihood, it doesn’t have an actual specific image of Noah to work from, but a web of concepts, connections, correlations. It’s probably analogous to if you were asked to draw a picture in the style of Da Vinci. Most peoples’ brains can’t reproduce the exact image, but a low fidelity representation of it that they can fill in with imagination. If you run the same prompt multiple times you can start to see evidence of this. The images are similar but also clearly different, not at all a quick transformation of an existing image.

ars 4y ago
It was trained with 400 million images, not a trillion. That's 11 bytes per image, and since most images are quite duplicative that actually is enough to more or less store every image permutation.
- ryankrage77 4y ago
  So it can store a little over 400 million images that are 3 pixels by 3 pixels in black and white. Or 2 by 2 pixels with three colours.
  - visarga 4y ago
    It's not even pixels the model is storing ... it's gradients from the loss function. Gradients that have the shape of the network, not of the image it is training on.
  - ars 4y ago
    It doesn't have to store every image - most are essentially duplicates of each other and only serve to reinforce which is the most representative image when multiple choices are linked with some reference text. It only needs to store a fraction of the images to have a never ending supply of source material (I guess that's kind of obvious since this program works :)
    - dougmwne 4y ago
      But again, I think the idea that the model weights store the source images as if it were a library is fallacious. There. Is no known way to get it to spit back out a copy of a training image.
- synu 4y ago
  There is some flexible encoding of terms to images there, too, that would take up some space. Though I’m not really sure you could decompress the weights in some way to regenerate the 400 million images and metadata.
- dougmwne 4y ago
  Sorry it was not a trillion. It was first trained on 2 billion, then fine tuned on smaller sets of highly aesthetic images. https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/Stable_Diffusion_v1_Model_Card.md https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...

ars 4y ago

It was trained with 400 million images, not a trillion. That's 11 bytes per image, and since most images are quite duplicative that actually is enough to more or less store every image permutation.

ryankrage77 4y ago
So it can store a little over 400 million images that are 3 pixels by 3 pixels in black and white. Or 2 by 2 pixels with three colours.
- visarga 4y ago
  It's not even pixels the model is storing ... it's gradients from the loss function. Gradients that have the shape of the network, not of the image it is training on.
- ars 4y ago
  It doesn't have to store every image - most are essentially duplicates of each other and only serve to reinforce which is the most representative image when multiple choices are linked with some reference text. It only needs to store a fraction of the images to have a never ending supply of source material (I guess that's kind of obvious since this program works :)
  - dougmwne 4y ago
    But again, I think the idea that the model weights store the source images as if it were a library is fallacious. There. Is no known way to get it to spit back out a copy of a training image.
synu 4y ago
There is some flexible encoding of terms to images there, too, that would take up some space. Though I’m not really sure you could decompress the weights in some way to regenerate the 400 million images and metadata.
dougmwne 4y ago
Sorry it was not a trillion. It was first trained on 2 billion, then fine tuned on smaller sets of highly aesthetic images. https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/Stable_Diffusion_v1_Model_Card.md https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...