4 ms·

I'm surprised how coherent most of these drawings are, instead of some warped monstronsity like you see from deepdream or thisanimedoesnotexist. Most of them h

by Evidlo 5y ago

I'm surprised how coherent most of these drawings are, instead of some warped monstronsity like you see from deepdream or thisanimedoesnotexist. Most of them have a theme that makes sense, too.

Evidlo 5y ago
OK, it looks like there is still some weirdness if you look at it too closely, like extra fingers or gross faces.
- jfoster 5y ago
  Those sometimes happen when humans create art, too.
tempestn 5y ago
This comment in another thread suggests why that's the case: > In other text-to-image algorithms I'm familiar with (the ones you'll typically see passed around as colab notebooks that people post outputs from on Twitter), the basic idea is to encode the text, and then try to make an image that maximally matches that text encoding. But this maximization often leads to artifacts - if you ask for an image of a sunset, you'll often get multiple suns, because that's even more sunset-like. There's a lot of tricks and hacks to regularize the process so that it's not so aggressive, but it's always an uphill battle. > Here, they instead take the text embedding, use a trained model (what they call the 'prior') to predict the corresponding image embedding - this removes the dangerous maximization. Then, another trained model (the 'decoder') produces images from the predicted embedding. https://news.ycombinator.com/item?id=30933091 https://news.ycombinator.com/item?id=30933091
- drcode 5y ago
  I also read somewhere that this system has special logic added to it that judges how humans would aesthetically judge the final image, so in a way the impressive aesthetic qualities of these images isn't totally coincidental.
  - amilios 5y ago
    This makes sense, OpenAI has been experimenting with human-in-the-loop type refinement systems for GPT-3 (see InstructGPT https://openai.com/blog/instruction-following/ https://openai.com/blog/instruction-following/). It would make sense that they would use something similar for Dall-E 2 as well.

by Evidlo 5y ago

I'm surprised how coherent most of these drawings are, instead of some warped monstronsity like you see from deepdream or thisanimedoesnotexist. Most of them have a theme that makes sense, too.

Evidlo 5y ago

OK, it looks like there is still some weirdness if you look at it too closely, like extra fingers or gross faces.

jfoster 5y ago
Those sometimes happen when humans create art, too.

tempestn 5y ago

This comment in another thread suggests why that's the case: > In other text-to-image algorithms I'm familiar with (the ones you'll typically see passed around as colab notebooks that people post outputs from on Twitter), the basic idea is to encode the text, and then try to make an image that maximally matches that text encoding. But this maximization often leads to artifacts - if you ask for an image of a sunset, you'll often get multiple suns, because that's even more sunset-like. There's a lot of tricks and hacks to regularize the process so that it's not so aggressive, but it's always an uphill battle. > Here, they instead take the text embedding, use a trained model (what they call the 'prior') to predict the corresponding image embedding - this removes the dangerous maximization. Then, another trained model (the 'decoder') produces images from the predicted embedding. https://news.ycombinator.com/item?id=30933091 https://news.ycombinator.com/item?id=30933091

drcode 5y ago
I also read somewhere that this system has special logic added to it that judges how humans would aesthetically judge the final image, so in a way the impressive aesthetic qualities of these images isn't totally coincidental.
- amilios 5y ago
  This makes sense, OpenAI has been experimenting with human-in-the-loop type refinement systems for GPT-3 (see InstructGPT https://openai.com/blog/instruction-following/ https://openai.com/blog/instruction-following/). It would make sense that they would use something similar for Dall-E 2 as well.