3 ms·

This is not very substantial work. They used the API, gave it more challenging questions on 14 prompts, and judged whether it met expectations. I would have exp

This is not very substantial work. They used the API, gave it more challenging questions on 14 prompts, and judged whether it met expectations. I would have expected something much more intelligent from Aaronson, although this kind of light-weight analysis is par for the course with Marcus, who has been fighting his rearguard action against large parametric models for many years, and is not above motivated reasoning. It's like one of those open letters that people get roped into, only hosted on arXiv.

yboris 4y ago
I don't see much of a downside to a light, quick exploration of a new tool and a writeup shared publicly. While this could have been a (very long) blog post, it seems like arXiv is a good place for this "report of findings".
- freediver 4y ago
  In this case the only thing "light" is the paper itself and manner in which the authors scrutinized the work. The prompts themselves were very complex and thought through, probably a result of a lot of cherrypicking to find weaknesses in the model. Still, it is impressive how DALL-E dealt with them even in cases where it got it wrong.
- standardly 4y ago
  I agree. The title is "A very preliminary analysis of DALL-E 2". There is really nothing offensive about this publication.
fullshark 4y ago
It's nice to see some examples that haven't been filtered by the creators as being the best ones, to show how capable it really is. This paper (at least to me) makes me even more impressed by DALL-E 2, it hasn't parsed the input exactly but it's still incredibly impressive imo.
karpierz 4y ago
I wouldn't describe these as challenging, the prompts are fairly simple if you're capable of understanding pretty basic concepts. > A pear cut into seven pieces arranged in a ring. > A couple in formal evening wear going home get caught in a heavy downpour with no umbrellas The work successfully demonstrates that the API lacks basic conceptual understanding by example.

yboris 4y ago

I don't see much of a downside to a light, quick exploration of a new tool and a writeup shared publicly. While this could have been a (very long) blog post, it seems like arXiv is a good place for this "report of findings".

freediver 4y ago
In this case the only thing "light" is the paper itself and manner in which the authors scrutinized the work. The prompts themselves were very complex and thought through, probably a result of a lot of cherrypicking to find weaknesses in the model. Still, it is impressive how DALL-E dealt with them even in cases where it got it wrong.
standardly 4y ago
I agree. The title is "A very preliminary analysis of DALL-E 2". There is really nothing offensive about this publication.

fullshark 4y ago

It's nice to see some examples that haven't been filtered by the creators as being the best ones, to show how capable it really is. This paper (at least to me) makes me even more impressed by DALL-E 2, it hasn't parsed the input exactly but it's still incredibly impressive imo.

karpierz 4y ago

I wouldn't describe these as challenging, the prompts are fairly simple if you're capable of understanding pretty basic concepts. > A pear cut into seven pieces arranged in a ring. > A couple in formal evening wear going home get caught in a heavy downpour with no umbrellas The work successfully demonstrates that the API lacks basic conceptual understanding by example.