3 ms·

I didn’t get a negative tone from the paper, it seemed mostly positive to me , just calling out a handful of areas ( like relative positioning ) where Dall-e fa

I didn’t get a negative tone from the paper, it seemed mostly positive to me , just calling out a handful of areas ( like relative positioning ) where Dall-e fails .

joshcryer 4y ago
For me, I read the introduction and how they discuss how impressive DALL-E 2 is, and then they show very simple mistakes it makes and I want to be protective of DALL-E 2. I want to say "you did a good job!" That the paper provides "a clearer picture of what remains to be done" is very hard to accept, as all it does is show edge cases which are subjective at best. If anything the picture is less clear as they don't even try to form a hypothesis why DALL-E makes mistakes like these. One thing in particular I have noticed is that DALL-E has trouble producing action images. It may be because it has no sense of temporality and in those cases it could serve to run the parameters a bit longer using the same scene. But I am not an AI scientist so what do I know.
vletal 4y ago
> ... it is reasonable to question whether DALL-E 2 constitutes progress toward solving the deep challenges of commonsense reasoning, comprehension, reliability, and so forth that would be needed for a truly general-purpose AI ... I like how they pose this question as a bait to the abstract yet they do not even attempt to answer it. Instead they focus on general shortcomings of the model. Moreover, there is no proper conclusion which would discuss the findings. Given how outspoken Gary Marcus is on Twitter - criticising current advances in DL I would expected him to do a much better job publishing a document about it.
- garymarcus 4y ago
  i would be glad to do if given proper access, but had into very limited indirect access.
soraki_soladead 4y ago
Many of the perceived failures of DALL-E 2 are subjective and ambiguous (by the author's own interpretation in some cases!) and speak to the author's negative biases which are well known outside of this publication. To be clear, I'm not defending DALL-E 2. I'm criticizing a poorly written paper that was published to Arxiv to lend further credibility for a Twitter audience to substantiate a claim that the DALL-E 2 authors have not made but that Gary Marcus has a vested interest in perpetuating: > How much does DALL-E have to do with AGI? Maybe not so much, after all… A lesson in caveat emptor: - https://twitter.com/GaryMarcus/status/1521120022298464256 https://twitter.com/GaryMarcus/status/1521120022298464256 This should have been a blog post or Twitter thread like the dozen or so other experimentations people have done with the system.
- joshcryer 4y ago
  Yikes, that Twitter feed: "Thoughts and prayers for the deep learning fanboys," as if ones admiration for DALL-E 2's achievement is something to be mocked... I feel like this sentiment is the way we're going to get I Have no Mouth, and I Must Scream. There's just something in there about negative AI minimalists or AI alarmists (same coin different side).
- pixl97 4y ago
  Heh, so many of these people's reaction to 'we have not created AGI' seems to be mocking and pessimism that we ever will. My response to 'we have not created AGI' is "Thank goodness". I don't think we're ready for that yet.
- nomel 4y ago
  > Many of the perceived failures of DALL-E 2 are subjective and ambiguous Could you expand on this a bit? DALL-E provides graphical, interpretive, output meant for humans. Isn't that necessarily subjective? Any qualitative metric of DALL-E 2 will need to involve some aggregate of humans being subjective. The papers title is "A very preliminary analysis of DALL-E 2", so low data points/opinions/speculation/further questions should be expected.
- jmmcd 4y ago
  Marcus is only one of three. From what we know if Aaronson, I doubt that this was a Marcus-driven paper with the others just along for the ride. But I agree some of the intro has a Marcus ring to it.
chrisco255 4y ago
I'd be curious to compare the results of Dall-E 2's output vs a group of human artists each individually given the exact same text prompt (with no follow up clarification allowed) and asked to produce 3-5 drafts.

I didn’t get a negative tone from the paper, it seemed mostly positive to me , just calling out a handful of areas ( like relative positioning ) where Dall-e fails .

joshcryer 4y ago

For me, I read the introduction and how they discuss how impressive DALL-E 2 is, and then they show very simple mistakes it makes and I want to be protective of DALL-E 2. I want to say "you did a good job!" That the paper provides "a clearer picture of what remains to be done" is very hard to accept, as all it does is show edge cases which are subjective at best. If anything the picture is less clear as they don't even try to form a hypothesis why DALL-E makes mistakes like these. One thing in particular I have noticed is that DALL-E has trouble producing action images. It may be because it has no sense of temporality and in those cases it could serve to run the parameters a bit longer using the same scene. But I am not an AI scientist so what do I know.

vletal 4y ago

> ... it is reasonable to question whether DALL-E 2 constitutes progress toward solving the deep challenges of commonsense reasoning, comprehension, reliability, and so forth that would be needed for a truly general-purpose AI ... I like how they pose this question as a bait to the abstract yet they do not even attempt to answer it. Instead they focus on general shortcomings of the model. Moreover, there is no proper conclusion which would discuss the findings. Given how outspoken Gary Marcus is on Twitter - criticising current advances in DL I would expected him to do a much better job publishing a document about it.

garymarcus 4y ago
i would be glad to do if given proper access, but had into very limited indirect access.

soraki_soladead 4y ago

Many of the perceived failures of DALL-E 2 are subjective and ambiguous (by the author's own interpretation in some cases!) and speak to the author's negative biases which are well known outside of this publication. To be clear, I'm not defending DALL-E 2. I'm criticizing a poorly written paper that was published to Arxiv to lend further credibility for a Twitter audience to substantiate a claim that the DALL-E 2 authors have not made but that Gary Marcus has a vested interest in perpetuating: > How much does DALL-E have to do with AGI? Maybe not so much, after all… A lesson in caveat emptor: - https://twitter.com/GaryMarcus/status/1521120022298464256 https://twitter.com/GaryMarcus/status/1521120022298464256 This should have been a blog post or Twitter thread like the dozen or so other experimentations people have done with the system.

joshcryer 4y ago
Yikes, that Twitter feed: "Thoughts and prayers for the deep learning fanboys," as if ones admiration for DALL-E 2's achievement is something to be mocked... I feel like this sentiment is the way we're going to get I Have no Mouth, and I Must Scream. There's just something in there about negative AI minimalists or AI alarmists (same coin different side).
pixl97 4y ago
Heh, so many of these people's reaction to 'we have not created AGI' seems to be mocking and pessimism that we ever will. My response to 'we have not created AGI' is "Thank goodness". I don't think we're ready for that yet.
nomel 4y ago
> Many of the perceived failures of DALL-E 2 are subjective and ambiguous Could you expand on this a bit? DALL-E provides graphical, interpretive, output meant for humans. Isn't that necessarily subjective? Any qualitative metric of DALL-E 2 will need to involve some aggregate of humans being subjective. The papers title is "A very preliminary analysis of DALL-E 2", so low data points/opinions/speculation/further questions should be expected.
jmmcd 4y ago
Marcus is only one of three. From what we know if Aaronson, I doubt that this was a Marcus-driven paper with the others just along for the ride. But I agree some of the intro has a Marcus ring to it.

chrisco255 4y ago

I'd be curious to compare the results of Dall-E 2's output vs a group of human artists each individually given the exact same text prompt (with no follow up clarification allowed) and asked to produce 3-5 drafts.