2 ms·
The biggest problem I see with LLM-generated imagery is a near total inability to get details right, which makes perfect sense when one considers how they work.
by jwells89 3y ago
The biggest problem I see with LLM-generated imagery is a near total inability to get details right, which makes perfect sense when one considers how they work.
LLMs pick out patterns in the data they're trained on and then regurgitates them. This works great for broad strokes, because those have relatively little variance between training pieces and have distinct visual signatures that act as anchors.
Details on the other hand differ dramatically between pieces and have no such consistent visual anchor. Take limbs for example, which are notoriously problematic for LLMs: there are so many different ways that arms, legs, and especially hands and fingers can look between their innumerable possible articulations, positions relative to the rest of the body, clothing, objects obscuring them, etc etc and the LLM, not actually understanding the subject matter, is predictably terrible at drawing the connections between all of these disparate states and struggles to draw them without human guidance.
You see this effect in other fine details, too. Jewelry, chain-link fences, fishing nets, chainmail, lace, etc are all near-guaranteed disasters for these things.
- orbital-decay 3y agoIt's mostly a problem of resolution, model size, and dataset quality, which can be mitigated with compositing. Larger models don't have problems with hands, and if they do, it can be solved by higher-order guidance (e.g. controlnets) and doing multiple supersampled passes on regions to avoid to fit too much detail in one generation. Even SD 1.5 (a notoriously tiny model) issues with faces and hands can be solved with multiple passes, which is what everyone does.