4 ms·

Regarding cherry-picking, the images of astronauts on horses look stunning, except for their hands. There's something seriously wrong with their hands. Maybe g

Regarding cherry-picking, the images of astronauts on horses look stunning, except for their hands. There's something seriously wrong with their hands. Maybe give it another five years, a few more $billion and a few more petabytes/flops and it will be good. Then finally everyone can generate art for their own Magic: the Gathering cards. (That's the end goal, right?)

a_bonobo 5y ago
Interestingly, hands are also something humans struggle to draw. They're a very complex anatomical form, many small tendons and muscles. Many artists struggle to depict hands. They're not made out of a few straight lines like a torso, there's lots of skew going on. They're probably the hardest structure of the human body to 'learn' for a ML system.
- matthew-wegner 5y ago
  I think some of this is because hands are very involved in both communication and threat assessment, so we as humans put a lot of automatic attention on them. We aren't even usually aware of it--unless something looks off Personally I really like leaning into the discontinuities and quirkiness of generated images. This is output from GLID-3: https://twitter.com/mwegner/status/1511139661095178241 https://twitter.com/mwegner/status/1511139661095178241
gwern 5y ago
As I keep telling people: "hands are hard". This is why I went so far as to make a hand-specific dataset ("PALM" https://www.gwern.net/Crops#palm https://www.gwern.net/Crops#palm which of course now everyone is going to confuse with 'PaLM'...). Hands are just way too variable to learn easily. My dataset is a start, but it may benefit from focused training, the way Facebook's new Make-A-Scene https://arxiv.org/abs/2203.13131#facebook https://arxiv.org/abs/2203.13131#facebook (not DALL-E 2 quality but not far from it) has focused losses on faces.
yokto 5y ago
Hands are notoriously hard to even photograph. You very quickly get weird unnatural results with a camera in front of hands, so in a way I'm not surprised AI models struggle to produce satisfying imagery there too.
MichaelDickens 5y ago
Just a thought, I wonder if DALL-E would generate more realistic hands if you prompted it with something like > an astronaut riding a horse, and the astronaut has five fingers on each hand

a_bonobo 5y ago

Interestingly, hands are also something humans struggle to draw. They're a very complex anatomical form, many small tendons and muscles. Many artists struggle to depict hands. They're not made out of a few straight lines like a torso, there's lots of skew going on. They're probably the hardest structure of the human body to 'learn' for a ML system.

matthew-wegner 5y ago
I think some of this is because hands are very involved in both communication and threat assessment, so we as humans put a lot of automatic attention on them. We aren't even usually aware of it--unless something looks off Personally I really like leaning into the discontinuities and quirkiness of generated images. This is output from GLID-3: https://twitter.com/mwegner/status/1511139661095178241 https://twitter.com/mwegner/status/1511139661095178241

gwern 5y ago

As I keep telling people: "hands are hard". This is why I went so far as to make a hand-specific dataset ("PALM" https://www.gwern.net/Crops#palm https://www.gwern.net/Crops#palm which of course now everyone is going to confuse with 'PaLM'...). Hands are just way too variable to learn easily. My dataset is a start, but it may benefit from focused training, the way Facebook's new Make-A-Scene https://arxiv.org/abs/2203.13131#facebook https://arxiv.org/abs/2203.13131#facebook (not DALL-E 2 quality but not far from it) has focused losses on faces.

yokto 5y ago

Hands are notoriously hard to even photograph. You very quickly get weird unnatural results with a camera in front of hands, so in a way I'm not surprised AI models struggle to produce satisfying imagery there too.

MichaelDickens 5y ago

Just a thought, I wonder if DALL-E would generate more realistic hands if you prompted it with something like > an astronaut riding a horse, and the astronaut has five fingers on each hand