4 ms·

The reason Imagen isn't made available to the public probably isn't about compositionality. The most notable thing about Alexander's challenge is that Imagen to

The reason Imagen isn't made available to the public probably isn't about compositionality. The most notable thing about Alexander's challenge is that Imagen totally failed every single one despite his claim of success because, apparently, it is programmed to never represent the human form. Not even Google employees are allowed to make it draw humans of any kind. They had to ask it to draw robots instead, but as pointed out in the comments, changing the requests in that way makes them much easier for DALL-E2 as well, especially the image with the top hats. If the creators have convinced themselves of some kind of "no humans" rule, but also know that this would be regarded as impossibly extreme and raise serious concerns about Google with the outside world, then keeping Imagen private forever may be the most "rational" solution.

adamsmith143 4y ago
>The most notable thing about Alexander's challenge is that Imagen totally failed every single one despite his claim of success because, apparently, it is programmed to never represent the human form. This doesn't make sense. The original challenge could well have been to draw robots to begin with. Has no bearing on the outcome imo.
- origin_path 4y ago
  But it wasn't, and it does make a difference. Dall-E really wants to draw top hats on people and not cats because the prompt is ambiguous and top hats are normally seen on humans so it struggles to overcome that bias. Neither robots not cats wear top hats so it's an easier problem to get right. But the real problem here is the refusal to do basic and normal things, like depict people. That's not normal - it's deeply weird and tells us a lot about what must be going on inside Google's ai research effort.
  - _dain_ 4y ago
    >But the real problem here is the refusal to do basic and normal things, like depict people. That's not normal - it's deeply weird and tells us a lot about what must be going on inside Google's ai research effort. Google is fighting a secret war against the Loab demon race that lives inside the high dimensional vector spaces. They've recently made incursions into our reality via Stable Diffusion.
  - adamsmith143 4y ago
    The inability to draw realistic humans is indeed strange but the question at hand is compositionality and so drawing a Robot with a top had is indeed more impressive precisely because it's not likely to be in the training data and shows a deeper understanding of the prompt. Presumably the model could randomly regurgitate a person with a top hat on that was seen in it's training data but that's not at all likely with a robot as you yourself said.
    - origin_path 4y ago
      It's not an inability, it's a policy choice, which is why it's weird. The question is why does Google think this rule is a good idea. Imagen could surely draw very good humans if allowed to. Robot looking at a cat wearing a top hat appears to be easier than with a human for DALL-E too, judging from the comments on Alexander's article, because both objects are neutral with respect to top hats. But really the whole set of prompts is poorly chosen. The original challenge of arbitrary shapes in relative positions seems the best way to test understanding of grammar and object relationships, exactly to avoid the "humans wear top hats and cats never do" problem. A better set of prompts is important - in this Gary Marcus is correct - exactly because there's no point defining a specific prompt if later you'll decide you accept a totally different prompt. That kind of invalidates the point of betting on well specified challenges to begin with.
jowday 4y ago
Imagen can produce images of humans - they’re just filtered out from the results by supervised models (for now). OpenAI did something similar with Dalle for a while IIRC.
- origin_path 4y ago
  That's a distinction without a difference and Dall-E will happily represent humans as long as they are "diverse".

adamsmith143 4y ago

>The most notable thing about Alexander's challenge is that Imagen totally failed every single one despite his claim of success because, apparently, it is programmed to never represent the human form. This doesn't make sense. The original challenge could well have been to draw robots to begin with. Has no bearing on the outcome imo.

origin_path 4y ago
But it wasn't, and it does make a difference. Dall-E really wants to draw top hats on people and not cats because the prompt is ambiguous and top hats are normally seen on humans so it struggles to overcome that bias. Neither robots not cats wear top hats so it's an easier problem to get right. But the real problem here is the refusal to do basic and normal things, like depict people. That's not normal - it's deeply weird and tells us a lot about what must be going on inside Google's ai research effort.
- _dain_ 4y ago
  >But the real problem here is the refusal to do basic and normal things, like depict people. That's not normal - it's deeply weird and tells us a lot about what must be going on inside Google's ai research effort. Google is fighting a secret war against the Loab demon race that lives inside the high dimensional vector spaces. They've recently made incursions into our reality via Stable Diffusion.
- adamsmith143 4y ago
  The inability to draw realistic humans is indeed strange but the question at hand is compositionality and so drawing a Robot with a top had is indeed more impressive precisely because it's not likely to be in the training data and shows a deeper understanding of the prompt. Presumably the model could randomly regurgitate a person with a top hat on that was seen in it's training data but that's not at all likely with a robot as you yourself said.
  - origin_path 4y ago
    It's not an inability, it's a policy choice, which is why it's weird. The question is why does Google think this rule is a good idea. Imagen could surely draw very good humans if allowed to. Robot looking at a cat wearing a top hat appears to be easier than with a human for DALL-E too, judging from the comments on Alexander's article, because both objects are neutral with respect to top hats. But really the whole set of prompts is poorly chosen. The original challenge of arbitrary shapes in relative positions seems the best way to test understanding of grammar and object relationships, exactly to avoid the "humans wear top hats and cats never do" problem. A better set of prompts is important - in this Gary Marcus is correct - exactly because there's no point defining a specific prompt if later you'll decide you accept a totally different prompt. That kind of invalidates the point of betting on well specified challenges to begin with.

jowday 4y ago

Imagen can produce images of humans - they’re just filtered out from the results by supervised models (for now). OpenAI did something similar with Dalle for a while IIRC.

origin_path 4y ago
That's a distinction without a difference and Dall-E will happily represent humans as long as they are "diverse".