9 ms·

A preliminary analysis of DALL-E 2 (Marcus, Davis, Aaronson)

71 points by gibsonf1 4y ago

AbrahamParangi 4y ago
To quote the generally incredible Scott Aaronson with a somewhat more optimistic perspective: Judged against where AI was 20-25 years ago, when I was a student, a dog is now holding meaningful conversations in English. And people are complaining that the dog isn’t a very eloquent orator, that it often makes grammatical errors and has to start again, that it took heroic effort to train it, and that it’s unclear how much the dog really understands.
visarga 4y ago
GPT-3 take on the paper abstract and title: > Reviewer #2: The authors provide a complement to the original DALL-E paper, but the paper is light on empirical evaluations. It is not clear why, for some of the prompts, the generated images are so poor and for others, they are pretty good. Why not provide a quantitative evaluation of this? (cherry picked from 10 attempts, temp=1)
MarcoZavala 4y ago
blueyes 4y ago
This is not very substantial work. They used the API, gave it more challenging questions on 14 prompts, and judged whether it met expectations. I would have expected something much more intelligent from Aaronson, although this kind of light-weight analysis is par for the course with Marcus, who has been fighting his rearguard action against large parametric models for many years, and is not above motivated reasoning. It's like one of those open letters that people get roped into, only hosted on arXiv.
- yboris 4y ago
  I don't see much of a downside to a light, quick exploration of a new tool and a writeup shared publicly. While this could have been a (very long) blog post, it seems like arXiv is a good place for this "report of findings".
  - freediver 4y ago
    In this case the only thing "light" is the paper itself and manner in which the authors scrutinized the work. The prompts themselves were very complex and thought through, probably a result of a lot of cherrypicking to find weaknesses in the model. Still, it is impressive how DALL-E dealt with them even in cases where it got it wrong.
  - standardly 4y ago
    I agree. The title is "A very preliminary analysis of DALL-E 2". There is really nothing offensive about this publication.
- fullshark 4y ago
  It's nice to see some examples that haven't been filtered by the creators as being the best ones, to show how capable it really is. This paper (at least to me) makes me even more impressed by DALL-E 2, it hasn't parsed the input exactly but it's still incredibly impressive imo.
- karpierz 4y ago
  I wouldn't describe these as challenging, the prompts are fairly simple if you're capable of understanding pretty basic concepts. > A pear cut into seven pieces arranged in a ring. > A couple in formal evening wear going home get caught in a heavy downpour with no umbrellas The work successfully demonstrates that the API lacks basic conceptual understanding by example.
soraki_soladead 4y ago
Gary Marcus here again to throw cold water on anything deep learning. The negative tone when not succeeding in tasks it wasn't designed or advertised for continues to be disappointing. > To the extent that the goal is to develop artificial intelligence that can be trusted in safety-critical applications (Marcus & Davis, 2019), a much higher standard must be applied. I have seen no mention of that being the intended use of DALL-E (2). In fact, the most common use case I've seen described is in replacing Fiverr-type tasks: quick graphic design. That said, the results are actually encouraging given that it _wasn't_ designed to succeed here: > Nevertheless, for 5 out of the 14 prompts, at least one of the ten images fully satisfied our requests. Some of the authors' interpretations could be argued against, as well. For instance, in example 10, "An old man is talking to his parents": > In none of these images did DALL-E successfully infer that image should show an old man with two even older people Several of the images appear to show exactly that? How is the author judging "even older"?
- garymarcus 4y ago
  How quickly they forget. 45 minutes after the DALL-E n release, Sam Altmann tweeted “AGI is gonna get wild”; Greg Brockman posted multiple tweets around the idea of AGI and walls broken. That invites a critical analysis. Ad hominem against me won’t remedy the limitations that we and others have observed.
  - awillen 4y ago
    This doesn't seem like an ad hominem attack at all. It's a criticism of criteria you're using to analyze DALL-E in this paper. You're talking about safety-critical applications, but nobody is expecting DALL-E to be used for those. Even your examples are just people tweeting generally about the future of AGI (the obvious context being that this is a big demonstrable step forward, not that DALL-E is an AGI that society is going to rely on for critical tasks). Sure, if you analyze it based on the criteria you've set out, it fails. The point is that nobody else cares if it fails based on that criteria, because nobody else thinks that criteria is relevant.
- rel2thr 4y ago
Imnimo 4y ago
It would be an interesting experiment to see where along the path some of these details are lost. For example, in the experiment where they ask for a pear cut into 7 slices, and Dall-E often gives the wrong number. First check if you can train a classifier on CLIP embeddings of the caption to recover the specified number. If so, check if you can do the same for the image embedding the prior produces. Then check if you can do it for unCLIP's internal feature maps, and so on. Similar experiments could be conducted for other lost attributes like object relationships (on top, behind, etc.) or specific identities (Lincoln, Washington, etc.) .
- visarga 4y ago
  Very good points, but I would start with the obvious one - the contrastive objective of CLIP forces the prompt representation to a single embedding vector. So it acts as a bottleneck, limiting the number of object-attribute pairs it can simultaneously encode. And of course the solution is to use an autoregressive setup like Dalle-1, but then they can't have the same diffusion model decoder for image generation. I bet they consciously prioritised the artistic applications over exact semantics. The CLIP embedding space has nice properties, it's tempting to use it. From my experience semantic similarity based on embedding dot products is much easier to do than exact semantic matching. Three is similar to four and red similar to blue in embedding space.
vintermann 4y ago
This is a very serious paper about a very serious topic, AI and AGI and how the... Who am I fooling? They're clearly just messing around and having fun. I don't begrudge them, though.
minimaxir 4y ago
This paper notes that average output can be flawed...which is the case with all AI-generated content and why it's important to note the importance of cherry picking/selection bias whenever posting it publicly. Fortunately, AI Image generation has helped visually communicate how AI content generation isn't quite sci-fi magic where you always get what you want with zero ambiguity. Yet.
- theptip 4y ago
  It’s an interesting point - cherry-picking muddies the water in an academic context. However it’s how these models will actually be used in the real world (with a human “editor” in the loop) so in some sense you care more about “best of N” performance. Or maybe even “best performance from someone that knows how to AI-whisper” since these are the new content creators that will produce the best output with these models (or their descendants). This is really a case where the real-world market will provide an assessment regardless of the sort of academic assessment of quality in the OP.
andybak 4y ago
Can someone remind of the link that turns arxiv papers into vaguely readable HTML?
- tonyg 4y ago
  https://ar5iv.labs.arxiv.org/ https://ar5iv.labs.arxiv.org/
- jwilk 4y ago
  https://www.arxiv-vanity.com/ https://www.arxiv-vanity.com/ but… "This paper doesn't have LaTeX source code, so it can't be rendered as a web page."
drcode 4y ago
Yes, DALL-E2 clearly gets confused when dealing with multiple objects, where each object is given specific attributes. However, it does answer SOME of these types of prompts correctly: It often correctly handles the simplest case of two objects with one specific property each. My prediction is that since it can handle SOME of these cases already, it means this problem has shown itself to be tractable, and future ML researchers will be able to chip away at the deficiencies with additional effort. I'm confident the next versions of these AI systems will handle the prompts given in this paper with ease.
- jandrese 4y ago
  From what I've read elsewhere DALL-E2 can't handle overly long/complex requests. Pretty much everybody who has tried it has discovered the same thing. Keep the requests simple.
- ravi-delia 4y ago
  I bet a lot of the issue is just the CLIP encoding. It's doing a ton of work getting text prompts into vector form, and it's incredible it does so well as is, but past a point you're squeezing blood from a stone. It's just not a full language model! Bump up the size, or figure something out about the space it's working in, I bet you'd see huge improvements.
Strilanc 4y ago
I really liked this paper. On the one hand, all they did was come up with some prompts and show the results. On the other hand, it does a really good job of being clear about exactly what they queried and why and how much selection they are applying (eg. they mention accidentally discarding a result once; this kind of thing was always very unclear in people's twitter posts of DALLE2 images). Also the paper highlights things that the model is clearly still struggling with, which researchers can then focus on improving.
- jmmcd 4y ago
  But I expected more clever reasoning and insight into what specific properties of prompts cause failures, in particular by trying variants of prompts. And more discussion of the weird style issue that causes "old man talking to his parents" to always be in a specific web-comic style.
  - astrange 4y ago
    They look a lot like the art from the copyright-free Japanese clip art site irasutoya. https://latelyjapanese.com/culture/20201214/do-you-know-about-irasutoyaいらすとや/ https://latelyjapanese.com/culture/20201214/do-you-know-abou...
shannifin 4y ago
Because many of the examples seem to show that the system easily misunderstands certain grammatical implications (prepositional relations, etc), I wonder if it would benefit from of different sort of input system in which the relationships of the objects in the desired image are more concretely defined, something more like a tree graph perhaps? Or perhaps this would be very difficult to implement? At the very least, seems it would be useful for users to edit a given output so that it can be continually tweaked, keeping what they like about an image (e.g. object layout, proportions, color scheme) while editing other aspects.
ridruejo 4y ago
The paper complains that some of the answers are not perfect but half the humans I know would have gotten them wrong… (thinking about the octopus one). I just find this technology fascinating. Imagine once you can extend this to video (“Star wars but the Jedis are the bad guys”, “breaking bad but an 80s sitcom” … Netflix should be investing in this…)
dukeofdoom 4y ago
How do you try this out? Is there a free alternative you can try? I would like to make some backgrounds for a platformer game.
dukeofdoom 4y ago
How do you try this out? Is there a free alternative you can try? I would like to make some backgrounds for a platformer game.
syspec 4y ago
Sorry for the n00b question, but how do I see the actual paper?
- gibsonf1 4y ago
  https://arxiv.org/pdf/2204.13807 https://arxiv.org/pdf/2204.13807
- sophiebits 4y ago
  Press the “Download PDF” button.
londons_explore 4y ago
Are these people just trying to pad their citation count? It really looks like nothing more than an hour's work to write this 'paper'.
- orbifold 4y ago
  Some academics are experts in also being science influencers. In those cases I actually know their non-scientific work better than their actual scientific contributions. That is I know Scott Aaronson from Shtetl optimised not from any particularly memorable scientific accomplishment, which he might well have.
  - jmmcd 4y ago
    He's surely a leading light in complexity, but since it's not my field I also know him more from the blog.
pjs_ 4y ago
Honestly try getting a bunch of PhDs in computer science to solve the same task and I think you will get dramatically worse results, not only artistically but also in terms of comprehension... The toaster/pyramid/ball one makes no sense to me - "with the pyramid behind a car that is above a toaster"?
Q6T46nT668w6i3m 4y ago
The criticism that autoregressive language models with billions of parameters don’t acquire basic logics (e.g., propositional, quantificational) is valid and, in my opinion, extremely interesting. Unfortunately, the subjectivity of their evaluation undermines that critique. The subjectivity is especially weird since these characteristics can be objectively evaluated (e.g., finding contradictions).
benlivengood 4y ago
The original paper examines some likely causes for failure to encode relational details between subjects, e.g. the corgi with a red hat and green necktie or vice versa chosen seemingly at random. The model seems to learn about contrasting subjects but not track particularly which attributes belong to which subjects. This is, as far as I can tell from the original paper, inherent in the training and would require different data sets or more parameters to learn a greater number of subject-attribute associations in each image. https://arxiv.org/abs/2204.06125 https://arxiv.org/abs/2204.06125
joering2 4y ago
> Caption: a red ball on top of a blue pyramid with the pyramid behind a car that is above a toaster. Have you ever heard humans talking like that? I bet you each person would also draw it differently. I consider myself human and yet I don't grasp a concept of a "car that is above toaster". What do they mean? Like hovering? Or just standing on the top? Cars aren't UFOs, they don't hover. The whole paper seems like they tried to break the system with tasks even humans would have problems grasping. What's the point??
- theptip 4y ago
  It’s like an IQ test. I think it’s a good way to probe the “cognition” (if you will) of the model. Sure, humans would probably differ in how they draw that picture. IQ 150 humans would almost all get the positional relationships correct though, even if the precise interpretation of “above” might differ. The fact that none of the images were correct under any interpretation of the relationships shows something meaningful, namely that this model can’t understand long chains of relationships.

71 points by gibsonf1 4y ago

To quote the generally incredible Scott Aaronson with a somewhat more optimistic perspective: Judged against where AI was 20-25 years ago, when I was a student, a dog is now holding meaningful conversations in English. And people are complaining that the dog isn’t a very eloquent orator, that it often makes grammatical errors and has to start again, that it took heroic effort to train it, and that it’s unclear how much the dog really understands.

visarga 4y ago

GPT-3 take on the paper abstract and title: > Reviewer #2: The authors provide a complement to the original DALL-E paper, but the paper is light on empirical evaluations. It is not clear why, for some of the prompts, the generated images are so poor and for others, they are pretty good. Why not provide a quantitative evaluation of this? (cherry picked from 10 attempts, temp=1)

MarcoZavala 4y ago

blueyes 4y ago

This is not very substantial work. They used the API, gave it more challenging questions on 14 prompts, and judged whether it met expectations. I would have expected something much more intelligent from Aaronson, although this kind of light-weight analysis is par for the course with Marcus, who has been fighting his rearguard action against large parametric models for many years, and is not above motivated reasoning. It's like one of those open letters that people get roped into, only hosted on arXiv.

yboris 4y ago
I don't see much of a downside to a light, quick exploration of a new tool and a writeup shared publicly. While this could have been a (very long) blog post, it seems like arXiv is a good place for this "report of findings".
- freediver 4y ago
  In this case the only thing "light" is the paper itself and manner in which the authors scrutinized the work. The prompts themselves were very complex and thought through, probably a result of a lot of cherrypicking to find weaknesses in the model. Still, it is impressive how DALL-E dealt with them even in cases where it got it wrong.
- standardly 4y ago
  I agree. The title is "A very preliminary analysis of DALL-E 2". There is really nothing offensive about this publication.
fullshark 4y ago
It's nice to see some examples that haven't been filtered by the creators as being the best ones, to show how capable it really is. This paper (at least to me) makes me even more impressed by DALL-E 2, it hasn't parsed the input exactly but it's still incredibly impressive imo.
karpierz 4y ago
I wouldn't describe these as challenging, the prompts are fairly simple if you're capable of understanding pretty basic concepts. > A pear cut into seven pieces arranged in a ring. > A couple in formal evening wear going home get caught in a heavy downpour with no umbrellas The work successfully demonstrates that the API lacks basic conceptual understanding by example.

soraki_soladead 4y ago

Gary Marcus here again to throw cold water on anything deep learning. The negative tone when not succeeding in tasks it wasn't designed or advertised for continues to be disappointing. > To the extent that the goal is to develop artificial intelligence that can be trusted in safety-critical applications (Marcus & Davis, 2019), a much higher standard must be applied. I have seen no mention of that being the intended use of DALL-E (2). In fact, the most common use case I've seen described is in replacing Fiverr-type tasks: quick graphic design. That said, the results are actually encouraging given that it _wasn't_ designed to succeed here: > Nevertheless, for 5 out of the 14 prompts, at least one of the ten images fully satisfied our requests. Some of the authors' interpretations could be argued against, as well. For instance, in example 10, "An old man is talking to his parents": > In none of these images did DALL-E successfully infer that image should show an old man with two even older people Several of the images appear to show exactly that? How is the author judging "even older"?

garymarcus 4y ago
How quickly they forget. 45 minutes after the DALL-E n release, Sam Altmann tweeted “AGI is gonna get wild”; Greg Brockman posted multiple tweets around the idea of AGI and walls broken. That invites a critical analysis. Ad hominem against me won’t remedy the limitations that we and others have observed.
- awillen 4y ago
  This doesn't seem like an ad hominem attack at all. It's a criticism of criteria you're using to analyze DALL-E in this paper. You're talking about safety-critical applications, but nobody is expecting DALL-E to be used for those. Even your examples are just people tweeting generally about the future of AGI (the obvious context being that this is a big demonstrable step forward, not that DALL-E is an AGI that society is going to rely on for critical tasks). Sure, if you analyze it based on the criteria you've set out, it fails. The point is that nobody else cares if it fails based on that criteria, because nobody else thinks that criteria is relevant.
rel2thr 4y ago

Imnimo 4y ago

It would be an interesting experiment to see where along the path some of these details are lost. For example, in the experiment where they ask for a pear cut into 7 slices, and Dall-E often gives the wrong number. First check if you can train a classifier on CLIP embeddings of the caption to recover the specified number. If so, check if you can do the same for the image embedding the prior produces. Then check if you can do it for unCLIP's internal feature maps, and so on. Similar experiments could be conducted for other lost attributes like object relationships (on top, behind, etc.) or specific identities (Lincoln, Washington, etc.) .

visarga 4y ago
Very good points, but I would start with the obvious one - the contrastive objective of CLIP forces the prompt representation to a single embedding vector. So it acts as a bottleneck, limiting the number of object-attribute pairs it can simultaneously encode. And of course the solution is to use an autoregressive setup like Dalle-1, but then they can't have the same diffusion model decoder for image generation. I bet they consciously prioritised the artistic applications over exact semantics. The CLIP embedding space has nice properties, it's tempting to use it. From my experience semantic similarity based on embedding dot products is much easier to do than exact semantic matching. Three is similar to four and red similar to blue in embedding space.

vintermann 4y ago

This is a very serious paper about a very serious topic, AI and AGI and how the... Who am I fooling? They're clearly just messing around and having fun. I don't begrudge them, though.

minimaxir 4y ago

This paper notes that average output can be flawed...which is the case with all AI-generated content and why it's important to note the importance of cherry picking/selection bias whenever posting it publicly. Fortunately, AI Image generation has helped visually communicate how AI content generation isn't quite sci-fi magic where you always get what you want with zero ambiguity. Yet.

theptip 4y ago
It’s an interesting point - cherry-picking muddies the water in an academic context. However it’s how these models will actually be used in the real world (with a human “editor” in the loop) so in some sense you care more about “best of N” performance. Or maybe even “best performance from someone that knows how to AI-whisper” since these are the new content creators that will produce the best output with these models (or their descendants). This is really a case where the real-world market will provide an assessment regardless of the sort of academic assessment of quality in the OP.

andybak 4y ago

Can someone remind of the link that turns arxiv papers into vaguely readable HTML?

tonyg 4y ago
https://ar5iv.labs.arxiv.org/ https://ar5iv.labs.arxiv.org/
jwilk 4y ago
https://www.arxiv-vanity.com/ https://www.arxiv-vanity.com/ but… "This paper doesn't have LaTeX source code, so it can't be rendered as a web page."

drcode 4y ago

Yes, DALL-E2 clearly gets confused when dealing with multiple objects, where each object is given specific attributes. However, it does answer SOME of these types of prompts correctly: It often correctly handles the simplest case of two objects with one specific property each. My prediction is that since it can handle SOME of these cases already, it means this problem has shown itself to be tractable, and future ML researchers will be able to chip away at the deficiencies with additional effort. I'm confident the next versions of these AI systems will handle the prompts given in this paper with ease.

jandrese 4y ago
From what I've read elsewhere DALL-E2 can't handle overly long/complex requests. Pretty much everybody who has tried it has discovered the same thing. Keep the requests simple.
ravi-delia 4y ago
I bet a lot of the issue is just the CLIP encoding. It's doing a ton of work getting text prompts into vector form, and it's incredible it does so well as is, but past a point you're squeezing blood from a stone. It's just not a full language model! Bump up the size, or figure something out about the space it's working in, I bet you'd see huge improvements.

Strilanc 4y ago

I really liked this paper. On the one hand, all they did was come up with some prompts and show the results. On the other hand, it does a really good job of being clear about exactly what they queried and why and how much selection they are applying (eg. they mention accidentally discarding a result once; this kind of thing was always very unclear in people's twitter posts of DALLE2 images). Also the paper highlights things that the model is clearly still struggling with, which researchers can then focus on improving.

jmmcd 4y ago
But I expected more clever reasoning and insight into what specific properties of prompts cause failures, in particular by trying variants of prompts. And more discussion of the weird style issue that causes "old man talking to his parents" to always be in a specific web-comic style.
- astrange 4y ago
  They look a lot like the art from the copyright-free Japanese clip art site irasutoya. https://latelyjapanese.com/culture/20201214/do-you-know-about-irasutoyaいらすとや/ https://latelyjapanese.com/culture/20201214/do-you-know-abou...

shannifin 4y ago

Because many of the examples seem to show that the system easily misunderstands certain grammatical implications (prepositional relations, etc), I wonder if it would benefit from of different sort of input system in which the relationships of the objects in the desired image are more concretely defined, something more like a tree graph perhaps? Or perhaps this would be very difficult to implement? At the very least, seems it would be useful for users to edit a given output so that it can be continually tweaked, keeping what they like about an image (e.g. object layout, proportions, color scheme) while editing other aspects.

ridruejo 4y ago

The paper complains that some of the answers are not perfect but half the humans I know would have gotten them wrong… (thinking about the octopus one). I just find this technology fascinating. Imagine once you can extend this to video (“Star wars but the Jedis are the bad guys”, “breaking bad but an 80s sitcom” … Netflix should be investing in this…)

dukeofdoom 4y ago

How do you try this out? Is there a free alternative you can try? I would like to make some backgrounds for a platformer game.

dukeofdoom 4y ago

How do you try this out? Is there a free alternative you can try? I would like to make some backgrounds for a platformer game.

syspec 4y ago

Sorry for the n00b question, but how do I see the actual paper?

gibsonf1 4y ago
https://arxiv.org/pdf/2204.13807 https://arxiv.org/pdf/2204.13807
sophiebits 4y ago
Press the “Download PDF” button.

londons_explore 4y ago

Are these people just trying to pad their citation count? It really looks like nothing more than an hour's work to write this 'paper'.

orbifold 4y ago
Some academics are experts in also being science influencers. In those cases I actually know their non-scientific work better than their actual scientific contributions. That is I know Scott Aaronson from Shtetl optimised not from any particularly memorable scientific accomplishment, which he might well have.
- jmmcd 4y ago
  He's surely a leading light in complexity, but since it's not my field I also know him more from the blog.

pjs_ 4y ago

Honestly try getting a bunch of PhDs in computer science to solve the same task and I think you will get dramatically worse results, not only artistically but also in terms of comprehension... The toaster/pyramid/ball one makes no sense to me - "with the pyramid behind a car that is above a toaster"?

Q6T46nT668w6i3m 4y ago

The criticism that autoregressive language models with billions of parameters don’t acquire basic logics (e.g., propositional, quantificational) is valid and, in my opinion, extremely interesting. Unfortunately, the subjectivity of their evaluation undermines that critique. The subjectivity is especially weird since these characteristics can be objectively evaluated (e.g., finding contradictions).

benlivengood 4y ago

The original paper examines some likely causes for failure to encode relational details between subjects, e.g. the corgi with a red hat and green necktie or vice versa chosen seemingly at random. The model seems to learn about contrasting subjects but not track particularly which attributes belong to which subjects. This is, as far as I can tell from the original paper, inherent in the training and would require different data sets or more parameters to learn a greater number of subject-attribute associations in each image. https://arxiv.org/abs/2204.06125 https://arxiv.org/abs/2204.06125

joering2 4y ago

> Caption: a red ball on top of a blue pyramid with the pyramid behind a car that is above a toaster. Have you ever heard humans talking like that? I bet you each person would also draw it differently. I consider myself human and yet I don't grasp a concept of a "car that is above toaster". What do they mean? Like hovering? Or just standing on the top? Cars aren't UFOs, they don't hover. The whole paper seems like they tried to break the system with tasks even humans would have problems grasping. What's the point??

theptip 4y ago
It’s like an IQ test. I think it’s a good way to probe the “cognition” (if you will) of the model. Sure, humans would probably differ in how they draw that picture. IQ 150 humans would almost all get the positional relationships correct though, even if the precise interpretation of “above” might differ. The fact that none of the images were correct under any interpretation of the relationships shows something meaningful, namely that this model can’t understand long chains of relationships.