6 ms·

Well, at some point it's like the monkeys writing shakespeare. Instead of a complicated neural network, we could have written a program that just outputs rando

by Imnimo 4y ago

Well, at some point it's like the monkeys writing shakespeare. Instead of a complicated neural network, we could have written a program that just outputs random pixel values. We'd definitely be able to find all of the outputs of Stable Diffusion, as well as all of the works of the original artists coming out of that program. It'd just take us a lot of waiting and watching. I don't know that that's enough to constitute "prior art" in any meaningful sense, though. The way I look at it is that the function of Stable Diffusion or any other diffusion model is to pare down the output space of the "random pixel machine". It learns which regions of image space are likely and which are unlikely, and so when you sample an image, you tend to get ones that people like rather than random noise. You could imagine an idealized diffusion process whose output space is truly continuous and which assigns non-zero probability to the entire image space (all possible pixel arrangements), with higher probability assigned to "good" regions, and lower probability assigned to "bad" regions. If I sample from such a model repeatedly, I will eventually (in the mathematical sense of probability 1 in the limit) get out an image that looks exactly like a Hollie Mengert illustration, even if the model has never seen one during training. I'll even eventually get out an image that looks exactly like an illustration that an unborn artist will create in the year 2047. Now, in practice, it's a little less clear. Are there regions of the image space that Stable Diffusion assigns exactly zero probability, such that no amount of sampling will ever generate that image? Are there real, "good" images that Stable Diffusion assigns very low probability such that generating them is no better than sampling from the "random pixel machine"?

doodlebugging 4y ago
>you tend to get ones that people like rather than random noise. And you have to consider that your idea of and tastes in "art" may be quite different from mine. People actually buy and display canvas with random bullshit colors or simple shapes or even lines on them that to me, is not art in the pure sense but is instead just the output of a lazy person marketed to dweebs who need something different for their living rooms. Back to the original discussion - if your earlier point is valid, I don't agree myself, then the only one who owns any art, product, process, etc is the one who designed that model that can be used to create it even if the model was created years after the thing they created. None of the people who actually created the product or process, etc own anything as a result of their efforts. That is wrong. It is a bit hand-wavey and bullshit mystic like the dumbass marketing used by some sculptors who claim they didn't really do anything except remove all the wood, stone, clay, etc that was hiding the sculpture they created. It was always right there carefully concealed in the tree trunk and anyone who dinked around with that trunk would've ended up creating the same sculpture under that logic. LOL. Jesus put it there, I'm only the guy picked to uncover it. >Are there real, "good" images that Stable Diffusion assigns very low probability such that generating them is no better than sampling from the "random pixel machine"? That depends on your definition of "good". Do you mean "good enough" that an observer will see a resemblance or is it something else? I would think that the output from any model will always have an upper limit to what it can reproduce that is related to the properties of the combined inputs to the model. With more inputs one should see higher precision outputs. Like you say though, Stable Diffusion probably designates some outputs as near zero probability because the input set supports that conclusion for the requested output.
- Imnimo 4y ago
  >I would think that the output from any model will always have an upper limit to what it can reproduce that is related to the properties of the combined inputs to the model. I look at it from the other direction. A diffusion model, at its heart, is a function that takes an image (initially random noise), and produce a slightly "better" image, according to its learned concept of "better". You turn this crank over and over and out comes a good image. But the simplest diffusion model of all is the identity function. You feed it random pixels and it outputs them unchanged. That model - the "random pixel machine" - can trivially output any possible image. The training process is paring down its output space to produce more outputs that are like the training images, and fewer outputs that are unlike them. So it's not the "upper limit" that training is addressing. Creating a model with an upper limit that includes all the great works of art humanity will ever create is trivial. It's the "lower limit" that's the problem - if those outputs are lost in a sea of uninteresting noise, you'll never find them. Training is the process of raising that lower limit.
  - doodlebugging 4y ago
    >according to its learned concept of "better". I'm gonna assume that all the input images fed to the process form it's "learned concept of better" (LCoB) so that in effect there is nothing random about the outputs. Indeed, each successive output becomes an optimization of the model fit to the "LCoB". Following that, it may start with an initial output that strongly resembles random noise but inside that first output will be a non-random component that initially fits some part of the LCoB model that it is trying to achieve. Also following that, the lower limit of the process is the output of the first step in the process since each successive iteration is an optimization towards a model. You already have the model noise floor when you run the first step. Struggling to take that lower is not smart and could be accomplished simply by excluding some part of the collection of images used to form it's LCoB. Does that make sense? There is nothing random about any of the outputs of this if it is model-driven. Stable Diffusion output is model driven therefore it is not random at any stage. In geophysical processing we have to carefully monitor outputs from each process to make sure that processing artifacts from mathematical operations can not create remnant waveforms that can be mistaken for geological data that could be used as a basis for drilling an expensive well. Data-creata is a real thing. Models are used. Thanks for the discussion.

by Imnimo 4y ago

doodlebugging 4y ago

>you tend to get ones that people like rather than random noise. And you have to consider that your idea of and tastes in "art" may be quite different from mine. People actually buy and display canvas with random bullshit colors or simple shapes or even lines on them that to me, is not art in the pure sense but is instead just the output of a lazy person marketed to dweebs who need something different for their living rooms. Back to the original discussion - if your earlier point is valid, I don't agree myself, then the only one who owns any art, product, process, etc is the one who designed that model that can be used to create it even if the model was created years after the thing they created. None of the people who actually created the product or process, etc own anything as a result of their efforts. That is wrong. It is a bit hand-wavey and bullshit mystic like the dumbass marketing used by some sculptors who claim they didn't really do anything except remove all the wood, stone, clay, etc that was hiding the sculpture they created. It was always right there carefully concealed in the tree trunk and anyone who dinked around with that trunk would've ended up creating the same sculpture under that logic. LOL. Jesus put it there, I'm only the guy picked to uncover it. >Are there real, "good" images that Stable Diffusion assigns very low probability such that generating them is no better than sampling from the "random pixel machine"? That depends on your definition of "good". Do you mean "good enough" that an observer will see a resemblance or is it something else? I would think that the output from any model will always have an upper limit to what it can reproduce that is related to the properties of the combined inputs to the model. With more inputs one should see higher precision outputs. Like you say though, Stable Diffusion probably designates some outputs as near zero probability because the input set supports that conclusion for the requested output.

Imnimo 4y ago
>I would think that the output from any model will always have an upper limit to what it can reproduce that is related to the properties of the combined inputs to the model. I look at it from the other direction. A diffusion model, at its heart, is a function that takes an image (initially random noise), and produce a slightly "better" image, according to its learned concept of "better". You turn this crank over and over and out comes a good image. But the simplest diffusion model of all is the identity function. You feed it random pixels and it outputs them unchanged. That model - the "random pixel machine" - can trivially output any possible image. The training process is paring down its output space to produce more outputs that are like the training images, and fewer outputs that are unlike them. So it's not the "upper limit" that training is addressing. Creating a model with an upper limit that includes all the great works of art humanity will ever create is trivial. It's the "lower limit" that's the problem - if those outputs are lost in a sea of uninteresting noise, you'll never find them. Training is the process of raising that lower limit.
- doodlebugging 4y ago
  >according to its learned concept of "better". I'm gonna assume that all the input images fed to the process form it's "learned concept of better" (LCoB) so that in effect there is nothing random about the outputs. Indeed, each successive output becomes an optimization of the model fit to the "LCoB". Following that, it may start with an initial output that strongly resembles random noise but inside that first output will be a non-random component that initially fits some part of the LCoB model that it is trying to achieve. Also following that, the lower limit of the process is the output of the first step in the process since each successive iteration is an optimization towards a model. You already have the model noise floor when you run the first step. Struggling to take that lower is not smart and could be accomplished simply by excluding some part of the collection of images used to form it's LCoB. Does that make sense? There is nothing random about any of the outputs of this if it is model-driven. Stable Diffusion output is model driven therefore it is not random at any stage. In geophysical processing we have to carefully monitor outputs from each process to make sure that processing artifacts from mathematical operations can not create remnant waveforms that can be mistaken for geological data that could be used as a basis for drilling an expensive well. Data-creata is a real thing. Models are used. Thanks for the discussion.