5 ms·

>you tend to get ones that people like rather than random noise. And you have to consider that your idea of and tastes in "art" may be quite different from min

>you tend to get ones that people like rather than random noise. And you have to consider that your idea of and tastes in "art" may be quite different from mine. People actually buy and display canvas with random bullshit colors or simple shapes or even lines on them that to me, is not art in the pure sense but is instead just the output of a lazy person marketed to dweebs who need something different for their living rooms. Back to the original discussion - if your earlier point is valid, I don't agree myself, then the only one who owns any art, product, process, etc is the one who designed that model that can be used to create it even if the model was created years after the thing they created. None of the people who actually created the product or process, etc own anything as a result of their efforts. That is wrong. It is a bit hand-wavey and bullshit mystic like the dumbass marketing used by some sculptors who claim they didn't really do anything except remove all the wood, stone, clay, etc that was hiding the sculpture they created. It was always right there carefully concealed in the tree trunk and anyone who dinked around with that trunk would've ended up creating the same sculpture under that logic. LOL. Jesus put it there, I'm only the guy picked to uncover it. >Are there real, "good" images that Stable Diffusion assigns very low probability such that generating them is no better than sampling from the "random pixel machine"? That depends on your definition of "good". Do you mean "good enough" that an observer will see a resemblance or is it something else? I would think that the output from any model will always have an upper limit to what it can reproduce that is related to the properties of the combined inputs to the model. With more inputs one should see higher precision outputs. Like you say though, Stable Diffusion probably designates some outputs as near zero probability because the input set supports that conclusion for the requested output.

Imnimo 4y ago
>I would think that the output from any model will always have an upper limit to what it can reproduce that is related to the properties of the combined inputs to the model. I look at it from the other direction. A diffusion model, at its heart, is a function that takes an image (initially random noise), and produce a slightly "better" image, according to its learned concept of "better". You turn this crank over and over and out comes a good image. But the simplest diffusion model of all is the identity function. You feed it random pixels and it outputs them unchanged. That model - the "random pixel machine" - can trivially output any possible image. The training process is paring down its output space to produce more outputs that are like the training images, and fewer outputs that are unlike them. So it's not the "upper limit" that training is addressing. Creating a model with an upper limit that includes all the great works of art humanity will ever create is trivial. It's the "lower limit" that's the problem - if those outputs are lost in a sea of uninteresting noise, you'll never find them. Training is the process of raising that lower limit.
- doodlebugging 4y ago
  >according to its learned concept of "better". I'm gonna assume that all the input images fed to the process form it's "learned concept of better" (LCoB) so that in effect there is nothing random about the outputs. Indeed, each successive output becomes an optimization of the model fit to the "LCoB". Following that, it may start with an initial output that strongly resembles random noise but inside that first output will be a non-random component that initially fits some part of the LCoB model that it is trying to achieve. Also following that, the lower limit of the process is the output of the first step in the process since each successive iteration is an optimization towards a model. You already have the model noise floor when you run the first step. Struggling to take that lower is not smart and could be accomplished simply by excluding some part of the collection of images used to form it's LCoB. Does that make sense? There is nothing random about any of the outputs of this if it is model-driven. Stable Diffusion output is model driven therefore it is not random at any stage. In geophysical processing we have to carefully monitor outputs from each process to make sure that processing artifacts from mathematical operations can not create remnant waveforms that can be mistaken for geological data that could be used as a basis for drilling an expensive well. Data-creata is a real thing. Models are used. Thanks for the discussion.
  - Imnimo 4y ago
    So the simplified view of a diffusion model is like the following (this leaves out the role of the prompt): -Sample random noise. -Ask the neural network "Here is a noisy image. What do you think it looked like before I added all this noise?" (note that you did not form this image by adding random noise to an initial image) -Adjust the pixels of your random noise towards what the network said. -Repeat until there is no noise left. During training, we take a training image and add noise to it. This way we know the "correct" (in scare quotes because there are many possible clean images that lead to the same noisy image given different realizations of the noise) answer to the question in step 2. This is used to update the weights of the neural network. Ultimately, a diffusion model is just a denoiser. A denoiser implicitly represents an underlying distribution of clean data. The diffusion process used to sample from the diffusion model is a clever way of drawing samples from that underlying distribution given access only to the denoiser. At sampling time, we have no training image that we add noise to. We just sample random noise out of thin air. This works because in the limit of large amounts of noise, the distribution of "initial image plus lots of noise" and "just lots of noise" are the same. You can certainly draw an analogy between this initial random noise and the uncarved block of marble that the sculptor says "contains" a sculpture waiting to be uncovered. Given the same noise, the neural network is deterministic - it will always produce the same output. You could even imagine an oracle who can unwind the process of the neural network being cranked and tell us exactly what initial noise sample would produce any desired output. Just like an artist might just draw the image that they want rather than waiting for the random pixel machine to output it, the oracle could simply set the precise values of Stable Diffusion's noise input to produce a Hollie Mengert work, rather than sampling repeatedly until it found one.

Imnimo 4y ago

>I would think that the output from any model will always have an upper limit to what it can reproduce that is related to the properties of the combined inputs to the model. I look at it from the other direction. A diffusion model, at its heart, is a function that takes an image (initially random noise), and produce a slightly "better" image, according to its learned concept of "better". You turn this crank over and over and out comes a good image. But the simplest diffusion model of all is the identity function. You feed it random pixels and it outputs them unchanged. That model - the "random pixel machine" - can trivially output any possible image. The training process is paring down its output space to produce more outputs that are like the training images, and fewer outputs that are unlike them. So it's not the "upper limit" that training is addressing. Creating a model with an upper limit that includes all the great works of art humanity will ever create is trivial. It's the "lower limit" that's the problem - if those outputs are lost in a sea of uninteresting noise, you'll never find them. Training is the process of raising that lower limit.

doodlebugging 4y ago
>according to its learned concept of "better". I'm gonna assume that all the input images fed to the process form it's "learned concept of better" (LCoB) so that in effect there is nothing random about the outputs. Indeed, each successive output becomes an optimization of the model fit to the "LCoB". Following that, it may start with an initial output that strongly resembles random noise but inside that first output will be a non-random component that initially fits some part of the LCoB model that it is trying to achieve. Also following that, the lower limit of the process is the output of the first step in the process since each successive iteration is an optimization towards a model. You already have the model noise floor when you run the first step. Struggling to take that lower is not smart and could be accomplished simply by excluding some part of the collection of images used to form it's LCoB. Does that make sense? There is nothing random about any of the outputs of this if it is model-driven. Stable Diffusion output is model driven therefore it is not random at any stage. In geophysical processing we have to carefully monitor outputs from each process to make sure that processing artifacts from mathematical operations can not create remnant waveforms that can be mistaken for geological data that could be used as a basis for drilling an expensive well. Data-creata is a real thing. Models are used. Thanks for the discussion.
- Imnimo 4y ago
  So the simplified view of a diffusion model is like the following (this leaves out the role of the prompt): -Sample random noise. -Ask the neural network "Here is a noisy image. What do you think it looked like before I added all this noise?" (note that you did not form this image by adding random noise to an initial image) -Adjust the pixels of your random noise towards what the network said. -Repeat until there is no noise left. During training, we take a training image and add noise to it. This way we know the "correct" (in scare quotes because there are many possible clean images that lead to the same noisy image given different realizations of the noise) answer to the question in step 2. This is used to update the weights of the neural network. Ultimately, a diffusion model is just a denoiser. A denoiser implicitly represents an underlying distribution of clean data. The diffusion process used to sample from the diffusion model is a clever way of drawing samples from that underlying distribution given access only to the denoiser. At sampling time, we have no training image that we add noise to. We just sample random noise out of thin air. This works because in the limit of large amounts of noise, the distribution of "initial image plus lots of noise" and "just lots of noise" are the same. You can certainly draw an analogy between this initial random noise and the uncarved block of marble that the sculptor says "contains" a sculpture waiting to be uncovered. Given the same noise, the neural network is deterministic - it will always produce the same output. You could even imagine an oracle who can unwind the process of the neural network being cranked and tell us exactly what initial noise sample would produce any desired output. Just like an artist might just draw the image that they want rather than waiting for the random pixel machine to output it, the oracle could simply set the precise values of Stable Diffusion's noise input to produce a Hollie Mengert work, rather than sampling repeatedly until it found one.