3 ms·
The >10 fingers problem should be solved via higher quality data or conditioning rather than a better model architecture.
by fzliu 3y ago
The >10 fingers problem should be solved via higher quality data or conditioning rather than a better model architecture.
- onion2k 3y agoAre you suggesting AI generates images of people with more than 10 fingers because there are too many pictures of people with more than 10 fingers in the input data? That seems unlikely..
- jobigoud 3y agoHigher quality could also mean the description attached to the original images in the dataset. If you were to describe a picture of someone you would never specifically call out the fact that they have 5 fingers per hand, we take it for granted, so that kind of information may never appear in the dataset. But I think what the grant parent means by conditioning is non-textual conditioning, like ControlNet. This will always be more powerful than trying to describe something by text. Think about the description of a character in a novel vs the movie adaptation.
- Drakim 3y agoIn some pictures not all 5 fingers on a hand are visible, so maybe it appears that humans have a variable amount of fingers?
- josefx 3y agoHumans with less than 5 fingers per hand do indeed exist. How does that lead to a default of 7?
- itishappy 3y agoCounting fingers is how humans do it, not necessarily how AI does. On a five-fingered hand, fingers are more likely to have neighbors than not. Why wouldn't the default be infinite fingers?
- Drakim 3y agoI meant more because people are holding objects or their hand is just at an angle where all fingers are not visible in the picture. I don't understand how neutral networks operate, but my layman's guess is that when you sometimes see hands with 5 fingers visible, sometimes 4, sometimes 3, sometimes 2, sometimes 1, and sometimes 0, then it's not immediately apparent that it means that every hand has between 5 and 0 fingers. Think of it this way, if the AI has ever only seen houses with a maximum of 10 windows in it's entire training set, is it so unthinkable that it sometimes draws a house with 12 windows? that's a "sensible" understanding about how houses have a variable amount of windows. It just doesn't work for fingers. I'm sure the same issue would arise if humans had other body parts that came in large quantities, but almost everything else is either 1 or 2 like the nose or eyes.
- notresidenter 3y agoPeople holding hands, people high-fiving, etc.
- amelius 3y agoYes, but if AI cannot solve the fingers problem, then can it reliably generate images of things which we have relatively few example images of? We have an enormous amount of images of hands.
- mdp2021 3y agoOutside of the work to study and improve available technologies ("how far can we push a hammer"), you do not draw an individual with anomalous hands because you have an ontological model in which "humans normally have five fingers per hand". Knowing "how the world works" is the appropriate source for subsequent expression of a representation. Somewhere in the architecture a world model should be formed.
- poulpy123 3y agoit's exactly the reverse: the issues with generative AI isn't the data anymore, but the models that are not able to understand the data