4 ms·
I'm a layman but the gist afaict is: These models don't understand relationships between objects in a scene, especially between distant objects. So they can't
by ye-olde-sysrq 4y ago
I'm a layman but the gist afaict is:
These models don't understand relationships between objects in a scene, especially between distant objects. So they can't do hands for the same reason they can't get legs on a table right. They know roughly what a table and a table leg look like, but they don't understand that there needs to be 3-4 of them at least, and they need to be spaced so that the table sits level, and the perspective they should have as a result. So, I've seen tables where it kind of gets it right that the legs are in the corners but then as the table legs go down, the front ones are mysteriously behind something that ought to be under the table. And sometimes it kind of loses track of a table leg or two - they melt into the background.
Very similar problem with hands. They need a very specific orientation and shape and the fingers all need to consistently point in the right direction, and typically the same direction (except for when they don't like with a pointed finger, etc).
Curious as to how these models handle it so much better than prior generations. Is it something novel, or a specific hand-based fix they put it, or is it just "we made the model bigger"?
- sho_hn 4y agoIt still feels unintuitive to me that models aren't able to infer these concepts from the training data given how consistently the training data follows them. It's not like there will be a lot of examples of bad hands in there.
- nashashmi 4y agoOr maybe the right models have not been built yet or plugged in? Another commenter told me about openpose information which is an AI that detects human poses. If that neuron is plugged in, it might lead to more accurate numbers. Stable diffusion is trying to do this.
- nuc1e0n 4y agoThere will be now though.
- stavros 4y agoMaybe the problem is that the model can't count, and just knows that each finger has a 75% chance to have another finger next to it.