5 ms·

That’s the interesting part: all the images generated are derived from a less than 4gb model (the trained weights of the neural network). So in a way, hundreds

That’s the interesting part: all the images generated are derived from a less than 4gb model (the trained weights of the neural network). So in a way, hundreds of billions of possible images are all stored in the model (each a vector in multidimensional latent space) and turned into pixels on demand (drived by the language model that knows how to turn words into a vector in this space) As it’s deterministic (given the exact same request parameters, random seed included, you get the exact same image) it’s a form of compression (or at least encoding decoding) too: I could send you the parameters for 1 million images that you would be able to recreate on your side, just as a relatively small text file.

amelius 4y ago
> As it’s deterministic (given the exact same request parameters, random seed included, you get the exact same image) it’s a form of compression (or at least encoding decoding) too: I could send you the parameters for 1 million images that you would be able to recreate on your side, just as a relatively small text file. For any input image? Or do you mean an image generated by the model?
- juliendorra 4y ago
  I meant images generated by the model. Now that I think of it I could just send you the sampled vectors and you could feed that to the vector to image part.
  - jimmySixDOF 4y ago
    My understanding is that images will not be bit identical due to GPU physics and decimal precision. Images from the same seed may be for all practical intents and purposes indistinguishable - but there are some flipped bits involved.
    - fragmede 4y ago
      That's not my understanding. The same seed value to the device's random number generator should results in the exact same outputs - there's a bug being chased down in the MPS (MacOS) backend where the fixed random seed doesn't output the same image on different computers.
      - squeaky-clean 4y ago
        I've heard something a bit in between what you're both saying. For the same machine with the same seed / parameters [0], your output is deterministic. But once you change hardware or OS you will probably get bit-level differences that won't make a macro-level difference. No idea how true that is, but on my windows machine, same params/seed is definitely deterministic. [0] a help string in the SD source code recommends the ddim_eta parameter (which isn't exposed in most web UI or GUI's, including the OP github) stay at the default 0.8 for deterministic sampling. I have no idea if this means changing the value from 0.8 produces non-deterministic results with the same hardware/os/params/seed. Or if they just mean changing this from 0.8 will make your SD not match the online model but still be deterministic itself. But in my testing, changing this value gives no useful changes to the image generation, so I keep it at 0.8
      - holoduke 4y ago
        If floats are used than there is no absolute deterministic behaviour accross different machines. It can never be guaranteed.
- based69 4y ago
  You could input an image and get it to recreate it as best as possible and then output a seed. That would be interesting!
  - ralusek 4y ago
    This is a fascinating idea. Have StableDiffusion generate an image from the image you'd like to "compress" + a random seed. Feed that output to an adversarial network that compares source image to output and scores it. Try again with new seed. After running for a while, the adversarial network outputs a seed, and you now have a few characters representing a reasonable approximation of your image.
    - mchusma 4y ago
      I expect something after jpegXL will be a neural network based compression scheme, where the client has a n GB neural net attached. There have been several that already show promising results (it's likely to be more of a standards issue than a technical issue).
  - holoduke 4y ago
    In 80s there was a man (forgot his name) who claimed that one day you could store an entire high res movie on a floppy disc. One day he might be right when AI can regenerate sequences of seeds to images/video. You just need a petabyte of models stored somewhere.
nope96 4y ago
So it's like a compiler which produces a 4GB executable file? And that 4GB is all the "logic" which can produce infinite possible images?
- MattRix 4y ago
  Not exactly. There’s no real logic per se, just data. It’s made up of tons of floating point numbers that define relationships to other floating point numbers.

amelius 4y ago

> As it’s deterministic (given the exact same request parameters, random seed included, you get the exact same image) it’s a form of compression (or at least encoding decoding) too: I could send you the parameters for 1 million images that you would be able to recreate on your side, just as a relatively small text file. For any input image? Or do you mean an image generated by the model?

juliendorra 4y ago
I meant images generated by the model. Now that I think of it I could just send you the sampled vectors and you could feed that to the vector to image part.
- jimmySixDOF 4y ago
  My understanding is that images will not be bit identical due to GPU physics and decimal precision. Images from the same seed may be for all practical intents and purposes indistinguishable - but there are some flipped bits involved.
  - fragmede 4y ago
    That's not my understanding. The same seed value to the device's random number generator should results in the exact same outputs - there's a bug being chased down in the MPS (MacOS) backend where the fixed random seed doesn't output the same image on different computers.
    - squeaky-clean 4y ago
      I've heard something a bit in between what you're both saying. For the same machine with the same seed / parameters [0], your output is deterministic. But once you change hardware or OS you will probably get bit-level differences that won't make a macro-level difference. No idea how true that is, but on my windows machine, same params/seed is definitely deterministic. [0] a help string in the SD source code recommends the ddim_eta parameter (which isn't exposed in most web UI or GUI's, including the OP github) stay at the default 0.8 for deterministic sampling. I have no idea if this means changing the value from 0.8 produces non-deterministic results with the same hardware/os/params/seed. Or if they just mean changing this from 0.8 will make your SD not match the online model but still be deterministic itself. But in my testing, changing this value gives no useful changes to the image generation, so I keep it at 0.8
    - holoduke 4y ago
      If floats are used than there is no absolute deterministic behaviour accross different machines. It can never be guaranteed.
based69 4y ago
You could input an image and get it to recreate it as best as possible and then output a seed. That would be interesting!
- ralusek 4y ago
  This is a fascinating idea. Have StableDiffusion generate an image from the image you'd like to "compress" + a random seed. Feed that output to an adversarial network that compares source image to output and scores it. Try again with new seed. After running for a while, the adversarial network outputs a seed, and you now have a few characters representing a reasonable approximation of your image.
  - mchusma 4y ago
    I expect something after jpegXL will be a neural network based compression scheme, where the client has a n GB neural net attached. There have been several that already show promising results (it's likely to be more of a standards issue than a technical issue).
- holoduke 4y ago
  In 80s there was a man (forgot his name) who claimed that one day you could store an entire high res movie on a floppy disc. One day he might be right when AI can regenerate sequences of seeds to images/video. You just need a petabyte of models stored somewhere.

nope96 4y ago

So it's like a compiler which produces a 4GB executable file? And that 4GB is all the "logic" which can produce infinite possible images?

MattRix 4y ago
Not exactly. There’s no real logic per se, just data. It’s made up of tons of floating point numbers that define relationships to other floating point numbers.