34 ms·

>according to its learned concept of "better". I'm gonna assume that all the input images fed to the process form it's "learned concept of better" (LCoB) so th

>according to its learned concept of "better". I'm gonna assume that all the input images fed to the process form it's "learned concept of better" (LCoB) so that in effect there is nothing random about the outputs. Indeed, each successive output becomes an optimization of the model fit to the "LCoB". Following that, it may start with an initial output that strongly resembles random noise but inside that first output will be a non-random component that initially fits some part of the LCoB model that it is trying to achieve. Also following that, the lower limit of the process is the output of the first step in the process since each successive iteration is an optimization towards a model. You already have the model noise floor when you run the first step. Struggling to take that lower is not smart and could be accomplished simply by excluding some part of the collection of images used to form it's LCoB. Does that make sense? There is nothing random about any of the outputs of this if it is model-driven. Stable Diffusion output is model driven therefore it is not random at any stage. In geophysical processing we have to carefully monitor outputs from each process to make sure that processing artifacts from mathematical operations can not create remnant waveforms that can be mistaken for geological data that could be used as a basis for drilling an expensive well. Data-creata is a real thing. Models are used. Thanks for the discussion.

Imnimo 4y ago
So the simplified view of a diffusion model is like the following (this leaves out the role of the prompt): -Sample random noise. -Ask the neural network "Here is a noisy image. What do you think it looked like before I added all this noise?" (note that you did not form this image by adding random noise to an initial image) -Adjust the pixels of your random noise towards what the network said. -Repeat until there is no noise left. During training, we take a training image and add noise to it. This way we know the "correct" (in scare quotes because there are many possible clean images that lead to the same noisy image given different realizations of the noise) answer to the question in step 2. This is used to update the weights of the neural network. Ultimately, a diffusion model is just a denoiser. A denoiser implicitly represents an underlying distribution of clean data. The diffusion process used to sample from the diffusion model is a clever way of drawing samples from that underlying distribution given access only to the denoiser. At sampling time, we have no training image that we add noise to. We just sample random noise out of thin air. This works because in the limit of large amounts of noise, the distribution of "initial image plus lots of noise" and "just lots of noise" are the same. You can certainly draw an analogy between this initial random noise and the uncarved block of marble that the sculptor says "contains" a sculpture waiting to be uncovered. Given the same noise, the neural network is deterministic - it will always produce the same output. You could even imagine an oracle who can unwind the process of the neural network being cranked and tell us exactly what initial noise sample would produce any desired output. Just like an artist might just draw the image that they want rather than waiting for the random pixel machine to output it, the oracle could simply set the precise values of Stable Diffusion's noise input to produce a Hollie Mengert work, rather than sampling repeatedly until it found one.
- doodlebugging 4y ago
  I think that we just described the same process. >-Sample random noise. >-Ask ... ...what the network said. >-Repeat until there is no noise left. Initially we have noise (random or not doesn't matter) and we are trying to find inside that noise a matching function to an image that we already have, our model or training image. That image may or may not also contain some residual noise and as you describe in your iterative steps, noise is added to it thus decreasing the signal to noise ratio, and the neural network compares the output of it's initial or updated image to the "correct answer" image and weights the next iteration accordingly so that a better match can be obtained. The function itself consists of an optimization process designed to minimize the residual noise (a de-noiser like you say) between the most recent image and our target image. When you say that you begin with a sample of random noise created by your array populating function which is assumed to be random but that you have no training image to which you are adding noise, that fits the whole process but it ignores that you are using the random noise image to iteratively produce an output that fits a model image created by compiling statistics from multiple images to create a target output that is assumed to be near noiseless or to have very high signal to noise ratio. If you are trying to use this to "find" Alfred E. Neuman in your output the process needs to know what Alfred E. Neuman looks like so it can effectively optimize to that result. Each iteration denoises based on the known output in the model created from the images that it must ingest in order to build the model. If you only have a few images of Alfred E. Neuman but you have thousands of Homer Simpson in your model input dataset, then you will have to fight through the tendency of the process to converge on Homer Simpson. No matter, you always have a priori information that is used to verify the integrity of the output. The input is irrelevant whether it is random or not since you are looking at an optimization process that matches, denoises, and weights iteratively until it minimizes an error function in the match and can be said to be an optimum, or good match. This is not particularly new or novel or anything else. It is a typical iterative modeling exercise like those that have been used for decades but now you have the compute power to build a near noise-free target model that fits the known data from every source at your disposal. The user who created the Hollie Mengert styled outputs could not have hit that target without using a model that was designed to create or mimic that type of output. That is why he chose to use her work in his process. He liked it. Then when he found out that she was not pleased about not being consulted and that she didn't have the rights to use some of those images then I think he had a come-to-Jesus moment that ultimately led him to rename it so he could feel better about it. Guilt-tripped him. Anyway, ethics should be a required part of every computer science curriculum especially when private personal information is involved. I'm in the oil and gas industry. It sucks sometimes. Fortunately there has been a push to include or require ethics training. Maybe one day it will clean up that industry. I'm only holding my breath though when I pass a refinery.

Imnimo 4y ago

So the simplified view of a diffusion model is like the following (this leaves out the role of the prompt): -Sample random noise. -Ask the neural network "Here is a noisy image. What do you think it looked like before I added all this noise?" (note that you did not form this image by adding random noise to an initial image) -Adjust the pixels of your random noise towards what the network said. -Repeat until there is no noise left. During training, we take a training image and add noise to it. This way we know the "correct" (in scare quotes because there are many possible clean images that lead to the same noisy image given different realizations of the noise) answer to the question in step 2. This is used to update the weights of the neural network. Ultimately, a diffusion model is just a denoiser. A denoiser implicitly represents an underlying distribution of clean data. The diffusion process used to sample from the diffusion model is a clever way of drawing samples from that underlying distribution given access only to the denoiser. At sampling time, we have no training image that we add noise to. We just sample random noise out of thin air. This works because in the limit of large amounts of noise, the distribution of "initial image plus lots of noise" and "just lots of noise" are the same. You can certainly draw an analogy between this initial random noise and the uncarved block of marble that the sculptor says "contains" a sculpture waiting to be uncovered. Given the same noise, the neural network is deterministic - it will always produce the same output. You could even imagine an oracle who can unwind the process of the neural network being cranked and tell us exactly what initial noise sample would produce any desired output. Just like an artist might just draw the image that they want rather than waiting for the random pixel machine to output it, the oracle could simply set the precise values of Stable Diffusion's noise input to produce a Hollie Mengert work, rather than sampling repeatedly until it found one.

doodlebugging 4y ago
I think that we just described the same process. >-Sample random noise. >-Ask ... ...what the network said. >-Repeat until there is no noise left. Initially we have noise (random or not doesn't matter) and we are trying to find inside that noise a matching function to an image that we already have, our model or training image. That image may or may not also contain some residual noise and as you describe in your iterative steps, noise is added to it thus decreasing the signal to noise ratio, and the neural network compares the output of it's initial or updated image to the "correct answer" image and weights the next iteration accordingly so that a better match can be obtained. The function itself consists of an optimization process designed to minimize the residual noise (a de-noiser like you say) between the most recent image and our target image. When you say that you begin with a sample of random noise created by your array populating function which is assumed to be random but that you have no training image to which you are adding noise, that fits the whole process but it ignores that you are using the random noise image to iteratively produce an output that fits a model image created by compiling statistics from multiple images to create a target output that is assumed to be near noiseless or to have very high signal to noise ratio. If you are trying to use this to "find" Alfred E. Neuman in your output the process needs to know what Alfred E. Neuman looks like so it can effectively optimize to that result. Each iteration denoises based on the known output in the model created from the images that it must ingest in order to build the model. If you only have a few images of Alfred E. Neuman but you have thousands of Homer Simpson in your model input dataset, then you will have to fight through the tendency of the process to converge on Homer Simpson. No matter, you always have a priori information that is used to verify the integrity of the output. The input is irrelevant whether it is random or not since you are looking at an optimization process that matches, denoises, and weights iteratively until it minimizes an error function in the match and can be said to be an optimum, or good match. This is not particularly new or novel or anything else. It is a typical iterative modeling exercise like those that have been used for decades but now you have the compute power to build a near noise-free target model that fits the known data from every source at your disposal. The user who created the Hollie Mengert styled outputs could not have hit that target without using a model that was designed to create or mimic that type of output. That is why he chose to use her work in his process. He liked it. Then when he found out that she was not pleased about not being consulted and that she didn't have the rights to use some of those images then I think he had a come-to-Jesus moment that ultimately led him to rename it so he could feel better about it. Guilt-tripped him. Anyway, ethics should be a required part of every computer science curriculum especially when private personal information is involved. I'm in the oil and gas industry. It sucks sometimes. Fortunately there has been a push to include or require ethics training. Maybe one day it will clean up that industry. I'm only holding my breath though when I pass a refinery.