4 ms·

The text this generates reminds me of the kind of text you'd see in a dream. The images look real enough, and the text looks like words; but totally unreadable.

by icey 4y ago

The text this generates reminds me of the kind of text you'd see in a dream. The images look real enough, and the text looks like words; but totally unreadable. Kind of an uncomfortable feeling.

junga 4y ago
I think I've never read text while dreaming.
- nikonyrh 4y ago
  That, and looking at a clock is a great way to check whether you are in a dream or not.
- aetherson 4y ago
  I have. You can read things while dreaming. Individual words are fine. But, just like dreams themselves, if you read more than a sentence or two, your mind will be unable to really keep a plot. And if you look away and read again, the text will have changed.
jtsiskin 4y ago
I don’t think this is a coincidence
bryans 4y ago
It's fascinating that the system is able to generate such precise imagery, yet can't handle words at all -- it can only barely reproduce words you explicitly tell it to. But sometimes it'll adds words when you're really not expecting it, and like you said, it's uncomfortable. Like the worst uncanny valley I've ever experienced. It was so unsettling to see the "four seasons" and "periodic table" that I couldn't stop nervously laughing.
- TuringTest 4y ago
  I think the words examples provide insight into how the image generation works at several levels. You can clearly see [1] how the glyphs are created by approximately merging several letters in one place to create a new symbol, while maintaining the overall structure of words and paragraphs within a well-composed layout in the page. The periodic table [2] seems to be doing the same, where the layout is the structure of colored boxes and two sizes of letters within them. Image composition seems to be doing something similar, learning to associate concepts with their visual representations at the right level, and merging already-seen examples in the right proportions to create a novel image. Cats and vampires are represented by their distinctive features, arms and legs are correctly positioned as parts of the body according to the action they perform, and instructions of style (either by artists like "Pieter Brueghel", art styles like "digital" or "mosaic", or even camera settings like ISO exposure [3]) are translated into lower level pixel representations of color, shapes and shadowing. My hypothesis is that if you included examples where the letters are taught one by one, like in kindergarten primers, it may be able to learn those concepts as well and generate better painted text (although I'm not sure it could make the jump to "understanding" the relation between their role as input instructions and as output image text). [1] https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/855638801ffcab0188609438f8c5e20aae809d060078a505.png/w_1024 https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/855... [2] https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/15713cb1e41bc072449c572342d5c2767d1b5f586afe11bb.png/w_1024 https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/157... [3] https://www.bramadams.dev/projects/dalle-tricks#let-there-be-lighting https://www.bramadams.dev/projects/dalle-tricks#let-there-be...
assbuttbuttass 4y ago
I was thinking the same thing. I commented elsewhere, but the way the AI messes up hands is also exactly like a dream
spyremeown 4y ago
Maybe… an uncanny feeling? These fall into The Valley for me. I don’t know why, but they do.

by icey 4y ago

The text this generates reminds me of the kind of text you'd see in a dream. The images look real enough, and the text looks like words; but totally unreadable. Kind of an uncomfortable feeling.

junga 4y ago

I think I've never read text while dreaming.

nikonyrh 4y ago
That, and looking at a clock is a great way to check whether you are in a dream or not.
aetherson 4y ago
I have. You can read things while dreaming. Individual words are fine. But, just like dreams themselves, if you read more than a sentence or two, your mind will be unable to really keep a plot. And if you look away and read again, the text will have changed.

jtsiskin 4y ago

I don’t think this is a coincidence

bryans 4y ago

It's fascinating that the system is able to generate such precise imagery, yet can't handle words at all -- it can only barely reproduce words you explicitly tell it to. But sometimes it'll adds words when you're really not expecting it, and like you said, it's uncomfortable. Like the worst uncanny valley I've ever experienced. It was so unsettling to see the "four seasons" and "periodic table" that I couldn't stop nervously laughing.

TuringTest 4y ago
I think the words examples provide insight into how the image generation works at several levels. You can clearly see [1] how the glyphs are created by approximately merging several letters in one place to create a new symbol, while maintaining the overall structure of words and paragraphs within a well-composed layout in the page. The periodic table [2] seems to be doing the same, where the layout is the structure of colored boxes and two sizes of letters within them. Image composition seems to be doing something similar, learning to associate concepts with their visual representations at the right level, and merging already-seen examples in the right proportions to create a novel image. Cats and vampires are represented by their distinctive features, arms and legs are correctly positioned as parts of the body according to the action they perform, and instructions of style (either by artists like "Pieter Brueghel", art styles like "digital" or "mosaic", or even camera settings like ISO exposure [3]) are translated into lower level pixel representations of color, shapes and shadowing. My hypothesis is that if you included examples where the letters are taught one by one, like in kindergarten primers, it may be able to learn those concepts as well and generate better painted text (although I'm not sure it could make the jump to "understanding" the relation between their role as input instructions and as output image text). [1] https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/855638801ffcab0188609438f8c5e20aae809d060078a505.png/w_1024 https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/855... [2] https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/15713cb1e41bc072449c572342d5c2767d1b5f586afe11bb.png/w_1024 https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/157... [3] https://www.bramadams.dev/projects/dalle-tricks#let-there-be-lighting https://www.bramadams.dev/projects/dalle-tricks#let-there-be...

assbuttbuttass 4y ago

I was thinking the same thing. I commented elsewhere, but the way the AI messes up hands is also exactly like a dream

spyremeown 4y ago

Maybe… an uncanny feeling? These fall into The Valley for me. I don’t know why, but they do.