3 ms·

I love this, and I have been enjoying DALL-E quite a bit, but does anyone else feel like AI art has a distinct look to it? I can't really put my finger on it, b

I love this, and I have been enjoying DALL-E quite a bit, but does anyone else feel like AI art has a distinct look to it? I can't really put my finger on it, but it invokes an almost unsettling feeling that I don't get from traditional artwork.

deleted 4y ago
[deleted]
ars 4y ago
It's because it's easy to see how it takes an existing artwork and modifies it. For example when asked to paint Noah it used a stereotypical image of last century Jews. How would it know that Noah is linked with Jews, if not by finding another painting labeled Noah and working off of that?
- dougmwne 4y ago
  This is where it gets interesting. The model is just 4.5gb. That’s too small to contain the actual trillion images it was trained on. So in all likelihood, it doesn’t have an actual specific image of Noah to work from, but a web of concepts, connections, correlations. It’s probably analogous to if you were asked to draw a picture in the style of Da Vinci. Most peoples’ brains can’t reproduce the exact image, but a low fidelity representation of it that they can fill in with imagination. If you run the same prompt multiple times you can start to see evidence of this. The images are similar but also clearly different, not at all a quick transformation of an existing image.
  - ars 4y ago
    It was trained with 400 million images, not a trillion. That's 11 bytes per image, and since most images are quite duplicative that actually is enough to more or less store every image permutation.
    - ryankrage77 4y ago
      So it can store a little over 400 million images that are 3 pixels by 3 pixels in black and white. Or 2 by 2 pixels with three colours.
      - visarga 4y ago
        It's not even pixels the model is storing ... it's gradients from the loss function. Gradients that have the shape of the network, not of the image it is training on.
      - ars 4y ago
        It doesn't have to store every image - most are essentially duplicates of each other and only serve to reinforce which is the most representative image when multiple choices are linked with some reference text. It only needs to store a fraction of the images to have a never ending supply of source material (I guess that's kind of obvious since this program works :)
        dougmwne 4y ago
        But again, I think the idea that the model weights store the source images as if it were a library is fallacious. There. Is no known way to get it to spit back out a copy of a training image.
    - synu 4y ago
      There is some flexible encoding of terms to images there, too, that would take up some space. Though I’m not really sure you could decompress the weights in some way to regenerate the 400 million images and metadata.
    - dougmwne 4y ago
      Sorry it was not a trillion. It was first trained on 2 billion, then fine tuned on smaller sets of highly aesthetic images. https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/Stable_Diffusion_v1_Model_Card.md https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...
- aaroninsf 4y ago
  Not to your point, I advocate for being more precise in the language we use when talking about how these systems function. These systems do not in any literal way "encode" or index or otherwise store specific images; they have incomprehensibly (to us) distributed representations representing aggregations over their training set. The thrust of the observation is correct; but the specifics are very different, it's probably a more accurate "lay person" description to say something like, "the system was trained with only a few references for what Noah looked like, and doesn't have much initiative by default to improvise or make inferences, so if it tries in good faith to render Noah it is limited by what it's seen." The point just being, it's less about making a collage or modifying a source, more about say 'doing its best to recapitulate a set of learned higher-level properties which are specific to the "Noah" concept.'
  - visarga 4y ago
    > it's less about making a collage or modifying a source, more about say 'doing its best to recapitulate a set of learned higher-level properties That sounds almost ... humane. I bet it will trigger some people who only see copyright violation in it. They don't understand the image is being created "from first principles".
bkandel 4y ago
For me a lot of it is in the eyes and face. There's almost always some weird deformation or artifact that doesn't look quite right. It also happens in most other pictures if you look hard enough but with eyes and faces it's really unnerving.
amelius 4y ago
Yes it's art that you get if you let a child make a broad sketch and let an experienced artist render all the details without looking at the whole result.
synu 4y ago
There are definitely some characteristic deformations that only AI does. Sort of cubist repeating of elements at transition points.
planetsprite 4y ago
I think it's the fact that you can subtly pick up the way Dall-e has begun to understand the similarity between different abstract forms. Dall-e has likely intuited that there is a general similarity between the faces of different animals, or a general similarity between different organic objects and different artificial ones. We are seeing the expression of those connections the model has made: compression in abstract representation that triggers the same uneasy feeling human brains get when on a bad trip and they see faces in clouds and shadows. The training of these image generation models minimizes the amount of necessary new visual information it needs to remember with each new image it sees during training, so whenever it sees a new image-caption pair it abstracts its qualities into an amalgamation of what its already seen. This is how Stable Diffusion managed to fit the complete corpus of human visual memory into 4.2 gigabytes. Everything is more closely connection than the picture would imply at first glace, like how in Super Mario Bros. the bushes are just clouds colored green. Compression and efficiency leads to a hyperconnectivity in the way DALL-E ties prompts and images. This is why those old deep-dream images where everything looked like a dog's face were so unsettling. This triggers an uncanny response in the human brain, because it sees the vague shape of a face in a tree, or the vague shape of a body in the structure of a building.

deleted 4y ago

[deleted]

ars 4y ago

It's because it's easy to see how it takes an existing artwork and modifies it. For example when asked to paint Noah it used a stereotypical image of last century Jews. How would it know that Noah is linked with Jews, if not by finding another painting labeled Noah and working off of that?

dougmwne 4y ago
This is where it gets interesting. The model is just 4.5gb. That’s too small to contain the actual trillion images it was trained on. So in all likelihood, it doesn’t have an actual specific image of Noah to work from, but a web of concepts, connections, correlations. It’s probably analogous to if you were asked to draw a picture in the style of Da Vinci. Most peoples’ brains can’t reproduce the exact image, but a low fidelity representation of it that they can fill in with imagination. If you run the same prompt multiple times you can start to see evidence of this. The images are similar but also clearly different, not at all a quick transformation of an existing image.
- ars 4y ago
  It was trained with 400 million images, not a trillion. That's 11 bytes per image, and since most images are quite duplicative that actually is enough to more or less store every image permutation.
  - ryankrage77 4y ago
    So it can store a little over 400 million images that are 3 pixels by 3 pixels in black and white. Or 2 by 2 pixels with three colours.
    - visarga 4y ago
      It's not even pixels the model is storing ... it's gradients from the loss function. Gradients that have the shape of the network, not of the image it is training on.
    - ars 4y ago
      It doesn't have to store every image - most are essentially duplicates of each other and only serve to reinforce which is the most representative image when multiple choices are linked with some reference text. It only needs to store a fraction of the images to have a never ending supply of source material (I guess that's kind of obvious since this program works :)
      - dougmwne 4y ago
        But again, I think the idea that the model weights store the source images as if it were a library is fallacious. There. Is no known way to get it to spit back out a copy of a training image.
  - synu 4y ago
    There is some flexible encoding of terms to images there, too, that would take up some space. Though I’m not really sure you could decompress the weights in some way to regenerate the 400 million images and metadata.
  - dougmwne 4y ago
    Sorry it was not a trillion. It was first trained on 2 billion, then fine tuned on smaller sets of highly aesthetic images. https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/Stable_Diffusion_v1_Model_Card.md https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...
aaroninsf 4y ago
Not to your point, I advocate for being more precise in the language we use when talking about how these systems function. These systems do not in any literal way "encode" or index or otherwise store specific images; they have incomprehensibly (to us) distributed representations representing aggregations over their training set. The thrust of the observation is correct; but the specifics are very different, it's probably a more accurate "lay person" description to say something like, "the system was trained with only a few references for what Noah looked like, and doesn't have much initiative by default to improvise or make inferences, so if it tries in good faith to render Noah it is limited by what it's seen." The point just being, it's less about making a collage or modifying a source, more about say 'doing its best to recapitulate a set of learned higher-level properties which are specific to the "Noah" concept.'
- visarga 4y ago
  > it's less about making a collage or modifying a source, more about say 'doing its best to recapitulate a set of learned higher-level properties That sounds almost ... humane. I bet it will trigger some people who only see copyright violation in it. They don't understand the image is being created "from first principles".

bkandel 4y ago

For me a lot of it is in the eyes and face. There's almost always some weird deformation or artifact that doesn't look quite right. It also happens in most other pictures if you look hard enough but with eyes and faces it's really unnerving.

amelius 4y ago

Yes it's art that you get if you let a child make a broad sketch and let an experienced artist render all the details without looking at the whole result.

synu 4y ago

There are definitely some characteristic deformations that only AI does. Sort of cubist repeating of elements at transition points.

planetsprite 4y ago

I think it's the fact that you can subtly pick up the way Dall-e has begun to understand the similarity between different abstract forms. Dall-e has likely intuited that there is a general similarity between the faces of different animals, or a general similarity between different organic objects and different artificial ones. We are seeing the expression of those connections the model has made: compression in abstract representation that triggers the same uneasy feeling human brains get when on a bad trip and they see faces in clouds and shadows. The training of these image generation models minimizes the amount of necessary new visual information it needs to remember with each new image it sees during training, so whenever it sees a new image-caption pair it abstracts its qualities into an amalgamation of what its already seen. This is how Stable Diffusion managed to fit the complete corpus of human visual memory into 4.2 gigabytes. Everything is more closely connection than the picture would imply at first glace, like how in Super Mario Bros. the bushes are just clouds colored green. Compression and efficiency leads to a hyperconnectivity in the way DALL-E ties prompts and images. This is why those old deep-dream images where everything looked like a dog's face were so unsettling. This triggers an uncanny response in the human brain, because it sees the vague shape of a face in a tree, or the vague shape of a body in the structure of a building.