4 ms·

I've been curious about the hand problem. It seems like it’s caused by the massive number of permutations possible with hands: all the joints, combined with al

I've been curious about the hand problem. It seems like it’s caused by the massive number of permutations possible with hands: all the joints, combined with all the angles at which they can be seen. I’ve wondered if this could be solved by creating a massive permutation of hand images using rigged 3D hand models (using, for example, Blender, or coupling with Unreal), and programmatically putting them in all possible combinations and angles possible from the rigging and then rendering millions of images of these combinations. Then the image models could learn from that artificially created dataset. Anyone with actual knowledge know how people are trying to tackle this?

Dalewyn 4y ago
It's really very simple: 1. Humans are terrible at drawing hands and feet (ask any illustrator, amateur or professional). 2. Advanced Interpolation software reads and generates images based off of both real-life photos and illustrations of hands drawn by humans, depending on what materials are fed to it. 3. The images generated carry forth the failings of humanity.
- aetherson 4y ago
  AI drawn hands are consistently terrible in different ways than bad human artists draw hands badly.
  - Dalewyn 4y ago
    As a man of culture who has witnessed his fair share of illustrated hands and feet: "AI" reflect human failings at drawing appendages very, very well.
    - kzrdude 4y ago
      Wrong number of fingers makes my skin crawl. I don't really have that problem with regular drawings.
IIAOPSW 4y ago
the entire "more data and pray" approach is how you's all end up with these problems. You want to see another stable diffusion generation failure, try asking Dalle or Craiyon for playing cards. In particular the back sides. Very similar to what you pointed out. Lots of repetition in the data sets, but non-exact repetitions. Different companies put different patterns and different stylistic choices on the same recognizable standard deck. The output the machine produces is frankly psychedelic. Here's what you missed. The generation problems all happen under a certain length scale. Its things under a certain band of fine detail where the distortion happens. Normally you won't notice it, because normally there isn't anything but noise in that band anyway. Shove a SD generated image through a fourier transform to see what I mean. And here's my very informed conjecture why it happens. The hint is right there in the name. Stable Diffusion. The generator network trains to de-noise images. The adversary adds Gaussian noise. That's a perfectly reasonable noise method, but it comes with its own spread and distribution in frequency space. A very similar problem exists with dithering processes. One way to use a b/w screen for grayscale is to treat the gray scale values as coin flip probabilities. This is known as random dithering. Its simple, its obvious, it should work, and it does work surprisingly well most of the time. But it runs into the exact same distribution problem. When there are closely spaced stripes in the image you want to show, that stripe pattern gets completely washed out by the noise being added to fake brightness levels. Put another way, overlaying TV static on any image makes it impossible to see blobs of the same size as might happen by random chance. The generator network can't see certain patterns, because the length scale of those patterns are coincident with the length scale of the noise being added by the adversary. This problem will never be solved by substituting data for comprehension. That's a one shot lesson worth generalizing.
- ChatGTP 4y ago
  Other commenters here seem to think MJ5 has solved the problem, are they wrong ? I don’t know just curious.
  - IIAOPSW 4y ago
    I just dug around to find those comments. The answer is I haven't seen MJ5 so I don't know, but when I see it I'll believe it. From other comments it seems MJ5 does better, but still has problems. I fully believe that all of these AI models can be ad hoc patched around the specific cases users complain about most, and that profitable companies desire a working product today more than a conceptual understanding that will pay off later. So I fully believe that there will be versions which "fix the hands problem", but the problem will still be there in at least some cases.
est 4y ago
> how people are trying to tackle this I don't think "people" actually tackle this. Ask a kid or non-professional to draw animal hands/feet without learning them first, e.g. crocodile hand, or feet of a seal. "People" ends up with uncanny drawings as well. There are tons of face photo on the Internet, but few of them are with hands clear visible.

Dalewyn 4y ago

It's really very simple: 1. Humans are terrible at drawing hands and feet (ask any illustrator, amateur or professional). 2. Advanced Interpolation software reads and generates images based off of both real-life photos and illustrations of hands drawn by humans, depending on what materials are fed to it. 3. The images generated carry forth the failings of humanity.

aetherson 4y ago
AI drawn hands are consistently terrible in different ways than bad human artists draw hands badly.
- Dalewyn 4y ago
  As a man of culture who has witnessed his fair share of illustrated hands and feet: "AI" reflect human failings at drawing appendages very, very well.
  - kzrdude 4y ago
    Wrong number of fingers makes my skin crawl. I don't really have that problem with regular drawings.

IIAOPSW 4y ago

the entire "more data and pray" approach is how you's all end up with these problems. You want to see another stable diffusion generation failure, try asking Dalle or Craiyon for playing cards. In particular the back sides. Very similar to what you pointed out. Lots of repetition in the data sets, but non-exact repetitions. Different companies put different patterns and different stylistic choices on the same recognizable standard deck. The output the machine produces is frankly psychedelic. Here's what you missed. The generation problems all happen under a certain length scale. Its things under a certain band of fine detail where the distortion happens. Normally you won't notice it, because normally there isn't anything but noise in that band anyway. Shove a SD generated image through a fourier transform to see what I mean. And here's my very informed conjecture why it happens. The hint is right there in the name. Stable Diffusion. The generator network trains to de-noise images. The adversary adds Gaussian noise. That's a perfectly reasonable noise method, but it comes with its own spread and distribution in frequency space. A very similar problem exists with dithering processes. One way to use a b/w screen for grayscale is to treat the gray scale values as coin flip probabilities. This is known as random dithering. Its simple, its obvious, it should work, and it does work surprisingly well most of the time. But it runs into the exact same distribution problem. When there are closely spaced stripes in the image you want to show, that stripe pattern gets completely washed out by the noise being added to fake brightness levels. Put another way, overlaying TV static on any image makes it impossible to see blobs of the same size as might happen by random chance. The generator network can't see certain patterns, because the length scale of those patterns are coincident with the length scale of the noise being added by the adversary. This problem will never be solved by substituting data for comprehension. That's a one shot lesson worth generalizing.

ChatGTP 4y ago
Other commenters here seem to think MJ5 has solved the problem, are they wrong ? I don’t know just curious.
- IIAOPSW 4y ago
  I just dug around to find those comments. The answer is I haven't seen MJ5 so I don't know, but when I see it I'll believe it. From other comments it seems MJ5 does better, but still has problems. I fully believe that all of these AI models can be ad hoc patched around the specific cases users complain about most, and that profitable companies desire a working product today more than a conceptual understanding that will pay off later. So I fully believe that there will be versions which "fix the hands problem", but the problem will still be there in at least some cases.

est 4y ago

> how people are trying to tackle this I don't think "people" actually tackle this. Ask a kid or non-professional to draw animal hands/feet without learning them first, e.g. crocodile hand, or feet of a seal. "People" ends up with uncanny drawings as well. There are tons of face photo on the Internet, but few of them are with hands clear visible.