4 ms·

That was an illustrative example don't take the trees literally. Here is an actual illustration of dithering which I believe is relevant. https://surma.dev/thi

That was an illustrative example don't take the trees literally. Here is an actual illustration of dithering which I believe is relevant. https://surma.dev/things/ditherpunk/ https://surma.dev/things/ditherpunk/ Imagine asking Dakke2 for a picture of the bridge through the fog. The problem is an image of fog is graphically indistinguishable from random noise. Whats the difference between a fog patch and something the adversary drew in? Good luck training out of that one. Less tortured understanding: think about my playing cards example. Consider a face down card with the ordinary geometric lacing patterns drawn on the back side. So we are all looking at the same thing: https://www.wopc.co.uk/images/countries/usa/standard/standard-pcc-2.jpg https://www.wopc.co.uk/images/countries/usa/standard/standar... Now think about what happens to that red card as I randomly add white noise static on top. The original fine patterning on the red side is soon distorted beyond recognition because those thin white lines are obscured by equiprobable white dots and coincidental random patterns.

GaggiX 4y ago
>Now think about what happens to that red card as I randomly add white noise static on top. The original fine patterning on the red side is soon distorted beyond recognition because those thin white lines are obscured by equiprobable white dots and coincidental random patterns. I don't see the problem, the pattern will remove the static white noise at the top and generate a new fine patterning on the red side (in case it did not go through the forward process to the end), of course it will not recover the lost information. >Imagine asking Dakke2 for a picture of the bridge through the fog. The problem is an image of fog is graphically indistinguishable from random noise. Whats the difference between a fog patch and something the adversary drew in? Good luck training out of that one. Ignoring the fact that Gaussian noise is really different from a patch of fog, it is obvious that if you have a high noise-to-signal ratio you will not be able to fully recover the signal, during the forward diffusion process all information is lost, however, the model will learn the distribution of samples. Also Dalle 2*
- IIAOPSW 4y ago
  >I don't see the problem, the pattern will remove the static white noise at the top and generate a new fine patterning on the red side (in case it did not go through the forward process to the end), of course it will not recover the lost information. You said it yourself, it erases information. Pivotally it isn't just a blunt eraser. It erases a very particular grain of detail. The problem is it looks like your on hallucinogenics. https://i.imgur.com/YlPHzgY.png https://i.imgur.com/YlPHzgY.png My hypothesis here is that the distortion manifests when you have fine grain patterns like you would find on a playing card because that is the length scale that is most affected by the noise process. The presence of noise doesn't effect long range objects. The same process would never cause the red card to conmpletely go away. It destroys information about that fine, thin, white line and patterns of a similar characteristic. That's really the key to what I'm saying.
  - GaggiX 4y ago
    >The same process would never cause the red card to conmpletely go away The forward diffusion process completely destroy all information, in fact when the model is trained we simply sample directly from the gaussian distribution instead of actually applying the forward diffusion process on a sample, so the fact that you lose information in the diffusion process is not really a problem because with the diffusion process you lose all information but the model clearly can still generate images.
    - IIAOPSW 4y ago
      I meant over a single step of the diffusion. Obviously if you keep adding noise it eventually all becomes noise. My point is the noise process is biased towards erasing some types of details faster than others. This works well when the length scale of features are proportional to the relevance and noticability, as is often the case (smaller size details usually matter less). But when the fine grain level actually has some sort of geometric patterning (such as the white lacing on the playing cards), stable diffusion can't even see it after the first steps of diffusion. It takes a lot more steps for the blob of red card overall to be eliminated than for the fine white lines. Its exactly the same sort of problem as in the dithering example. An original image of 2-pixel-thin stripes is very recognizable to us as a "thin striped object". Trying to color it in with random dithering will render it unrecognizable. However a much wider 10-pixel-stripe pattern can be safely colored with random dithering. The commonality is long range ordering of short range features. Thats what the SD process struggles with.

GaggiX 4y ago

>Now think about what happens to that red card as I randomly add white noise static on top. The original fine patterning on the red side is soon distorted beyond recognition because those thin white lines are obscured by equiprobable white dots and coincidental random patterns. I don't see the problem, the pattern will remove the static white noise at the top and generate a new fine patterning on the red side (in case it did not go through the forward process to the end), of course it will not recover the lost information. >Imagine asking Dakke2 for a picture of the bridge through the fog. The problem is an image of fog is graphically indistinguishable from random noise. Whats the difference between a fog patch and something the adversary drew in? Good luck training out of that one. Ignoring the fact that Gaussian noise is really different from a patch of fog, it is obvious that if you have a high noise-to-signal ratio you will not be able to fully recover the signal, during the forward diffusion process all information is lost, however, the model will learn the distribution of samples. Also Dalle 2*

IIAOPSW 4y ago
>I don't see the problem, the pattern will remove the static white noise at the top and generate a new fine patterning on the red side (in case it did not go through the forward process to the end), of course it will not recover the lost information. You said it yourself, it erases information. Pivotally it isn't just a blunt eraser. It erases a very particular grain of detail. The problem is it looks like your on hallucinogenics. https://i.imgur.com/YlPHzgY.png https://i.imgur.com/YlPHzgY.png My hypothesis here is that the distortion manifests when you have fine grain patterns like you would find on a playing card because that is the length scale that is most affected by the noise process. The presence of noise doesn't effect long range objects. The same process would never cause the red card to conmpletely go away. It destroys information about that fine, thin, white line and patterns of a similar characteristic. That's really the key to what I'm saying.
- GaggiX 4y ago
  >The same process would never cause the red card to conmpletely go away The forward diffusion process completely destroy all information, in fact when the model is trained we simply sample directly from the gaussian distribution instead of actually applying the forward diffusion process on a sample, so the fact that you lose information in the diffusion process is not really a problem because with the diffusion process you lose all information but the model clearly can still generate images.
  - IIAOPSW 4y ago
    I meant over a single step of the diffusion. Obviously if you keep adding noise it eventually all becomes noise. My point is the noise process is biased towards erasing some types of details faster than others. This works well when the length scale of features are proportional to the relevance and noticability, as is often the case (smaller size details usually matter less). But when the fine grain level actually has some sort of geometric patterning (such as the white lacing on the playing cards), stable diffusion can't even see it after the first steps of diffusion. It takes a lot more steps for the blob of red card overall to be eliminated than for the fine white lines. Its exactly the same sort of problem as in the dithering example. An original image of 2-pixel-thin stripes is very recognizable to us as a "thin striped object". Trying to color it in with random dithering will render it unrecognizable. However a much wider 10-pixel-stripe pattern can be safely colored with random dithering. The commonality is long range ordering of short range features. Thats what the SD process struggles with.