5 ms·

Another gallery with 80 ddim steps: https://postimg.cc/gallery/b1kn7yd https://postimg.cc/gallery/b1kn7yd Thought I'd share this for others interested. I've mo

Another gallery with 80 ddim steps: https://postimg.cc/gallery/b1kn7yd https://postimg.cc/gallery/b1kn7yd Thought I'd share this for others interested. I've modified txt2img to save an image after each step. (actually quite easy as you can specify an img_callback to sampler) Interestingly, both of these runs are using the same seed and prompt, yet they yield different final images, the only difference is the number of ddim sampling steps. I'd love to understand why if anyone has any idea.

sp332 4y ago
A couple of replies to https://news.ycombinator.com/item?id=32634807 https://news.ycombinator.com/item?id=32634807 suggest some sources of non-determinism.
- TheMiddleMan 4y ago
  Interesting. I suppose GPUs could calculate things differently. I just checked, I can rerun both 40/80 step runs and the final images are bit-identical to the first runs. So at least in my scenario the same parameters are deterministic, but changing the number of ddim sampling steps changes the result. Maybe it's doing something fancy with the total number of steps, beyond just stopping after the count is reached.
  - dangero 4y ago
    It will most likely render differently on different hardware since gpu float math is nondeterministic across different environments
    - jimmySixDOF 4y ago
      aaaand there goes people trying to turn this into a 99% compression system
    - lbotos 4y ago
      How are cloud GPU providers handling this then? Do the fancy A1000 chips solve this?
beecafe 4y ago
Each sampling step runs at a specific scale, fewer steps would skip some of the intermediate scales
- TheMiddleMan 4y ago
  Ah I see, it makes bigger leaps each step to try to get to the same end result in less total steps. That makes sense, assuming I have it right.
mpaepper 4y ago
Most GPUs are non-deterministic - learned this the hard way in deep learning on pathology data. This is for optimization purposes. In fact, you can set a flag in Pytorch / Cuda to disable this which comes at the cost of performance.
- deleted 4y ago
  [deleted]
- riedel 4y ago
  Can you explain? How much does it actually affect results in extreme cases? The source of non-determinism does seem the GPU but parallelism and dynamic allocation in the frameworks. (Also seems that some parts of pytorch still return runtime error if you request a deterministic version). Are there other more performant deterministic DL frameworks?
mgarciaisaia 4y ago
Do you have a diff/patch of the change to do this? I may try understanding both StableDiffusion and Python enough to do it, but if you already solved it - that'll be appreciated :)
- gbear605 4y ago
  You can set both the seed and the number of inference steps when running StableDiffusion locally (or in Google CoLab). I assumed that they just set a seed and then generated the image at each inference step. With a decent GPU, it’s only going to take a few minutes. You could definitely modify it to output at each step, but the output step takes a relatively long time, so it would slow down the process.

sp332 4y ago

A couple of replies to https://news.ycombinator.com/item?id=32634807 https://news.ycombinator.com/item?id=32634807 suggest some sources of non-determinism.

TheMiddleMan 4y ago
Interesting. I suppose GPUs could calculate things differently. I just checked, I can rerun both 40/80 step runs and the final images are bit-identical to the first runs. So at least in my scenario the same parameters are deterministic, but changing the number of ddim sampling steps changes the result. Maybe it's doing something fancy with the total number of steps, beyond just stopping after the count is reached.
- dangero 4y ago
  It will most likely render differently on different hardware since gpu float math is nondeterministic across different environments
  - jimmySixDOF 4y ago
    aaaand there goes people trying to turn this into a 99% compression system
  - lbotos 4y ago
    How are cloud GPU providers handling this then? Do the fancy A1000 chips solve this?

beecafe 4y ago

Each sampling step runs at a specific scale, fewer steps would skip some of the intermediate scales

TheMiddleMan 4y ago
Ah I see, it makes bigger leaps each step to try to get to the same end result in less total steps. That makes sense, assuming I have it right.

mpaepper 4y ago

Most GPUs are non-deterministic - learned this the hard way in deep learning on pathology data. This is for optimization purposes. In fact, you can set a flag in Pytorch / Cuda to disable this which comes at the cost of performance.

deleted 4y ago
[deleted]
riedel 4y ago
Can you explain? How much does it actually affect results in extreme cases? The source of non-determinism does seem the GPU but parallelism and dynamic allocation in the frameworks. (Also seems that some parts of pytorch still return runtime error if you request a deterministic version). Are there other more performant deterministic DL frameworks?

mgarciaisaia 4y ago

Do you have a diff/patch of the change to do this? I may try understanding both StableDiffusion and Python enough to do it, but if you already solved it - that'll be appreciated :)

gbear605 4y ago
You can set both the seed and the number of inference steps when running StableDiffusion locally (or in Google CoLab). I assumed that they just set a seed and then generated the image at each inference step. With a decent GPU, it’s only going to take a few minutes. You could definitely modify it to output at each step, but the output step takes a relatively long time, so it would slow down the process.