3 ms·

By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different? A fan artis

by cheald 3y ago

By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different? A fan artist must observe a character's official depiction before they can produce their own variants of it. Most humans do not produce art in a clean room, either - we all produce a synthesis of the existing art we've been exposed to. A LoRA or dreambooth model incorporating a novel concept is conceptually similar to a fan artist being shown a new character so they can produce new derivations of it. You could produce "police sketches" of a previously unseen character based on shared description, but it would likely fail to capture the nuance of the individual just as police sketches would.

jsheard 3y ago
> By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different? I've yet to see a LoRA produce a meaningfully diverse and high quality range of interpretations based on just a single piece of character key art, or brief appearance in a trailer, the way the conventional fan-artists do. What actually happens is that those conventional artists produce a wide variety of interpretations, someone scrapes those hundreds or thousands of images, and then trains a model using that dataset instead. The model needs the variety and novelty that the human artists bring to the table, otherwise it would just over-fit to the official art. When I was looking for examples of Zelda AI art I noticed there is a LoRA for Purah, and the first version was posted on CivetAI not when her TOTK design was revealed, nor when the game came out, but a month later when there was a wide enough corpus of existing fanart to scrape together a decent model with. It only works by piggybacking off the non-AI artists effort.
- cheald 3y ago
  I get what you mean, but my point is that fundamentally, what's happening in the LoRA and in the human artist is conceptually similar - each learns through examples what it is about a character that makes that character distinctive, and then attempts to reproduce it within some set of constraints. Neither is capable of doing so without first being exposed to what it is that makes that character distinctive. Human brains are obviously far better at interpretation than current AIs are (and have the advantage of actual will and intent), but I don't know why one'd expect a model that's never seen a character to be able to reproduce the essential nature of that character. I rather suspect that you can get more variation than you'd think from a homogenous dataset, though - I've trained a LoRA on myself that can produce photorealistic renderings of me, as well as anime, Disney, Pixar, and Dreamworks styles, all of which actually feel pretty "me"; I don't have any human-produced drawings of myself as a Disney character or renderings of me in a Pixar movie, but the model does a good job of projecting what I'd look like given what it knows about "Disney-style" drawings, plus what I've taught it about what I look like.
- cheald 3y ago
  For fun, I took two shots of Sonia from the third TOTK trailer (who should be novel and not in the model), trained a LoRA for 400 steps on just them (based on the AbyssOrange anime model), then threw it into an entirely different model (Dreamshaper, which mixes anime and realistic models) with some prompting to get something other than the original. https://imgur.com/a/iPNPojK https://imgur.com/a/iPNPojK I'm not gonna say that it's perfect, but as an interpretation of the character based on exactly two nearly-identical screen grabs trained for a whole 400 steps and some amateur prompting? Not too shabby at all. It hit many of the distinctive points of the character - darker skin, blonde hair with ringlets, the forehead emblem, the tear tattoos under the eyes, even the laurels under the hair. It missed the bell earrings and the secret stone necklace, but given the limited source material, I think it's quite good. With a few more varied screenshots, I'll bet that it could do quite impressively.

by cheald 3y ago

jsheard 3y ago

> By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different? I've yet to see a LoRA produce a meaningfully diverse and high quality range of interpretations based on just a single piece of character key art, or brief appearance in a trailer, the way the conventional fan-artists do. What actually happens is that those conventional artists produce a wide variety of interpretations, someone scrapes those hundreds or thousands of images, and then trains a model using that dataset instead. The model needs the variety and novelty that the human artists bring to the table, otherwise it would just over-fit to the official art. When I was looking for examples of Zelda AI art I noticed there is a LoRA for Purah, and the first version was posted on CivetAI not when her TOTK design was revealed, nor when the game came out, but a month later when there was a wide enough corpus of existing fanart to scrape together a decent model with. It only works by piggybacking off the non-AI artists effort.

cheald 3y ago
I get what you mean, but my point is that fundamentally, what's happening in the LoRA and in the human artist is conceptually similar - each learns through examples what it is about a character that makes that character distinctive, and then attempts to reproduce it within some set of constraints. Neither is capable of doing so without first being exposed to what it is that makes that character distinctive. Human brains are obviously far better at interpretation than current AIs are (and have the advantage of actual will and intent), but I don't know why one'd expect a model that's never seen a character to be able to reproduce the essential nature of that character. I rather suspect that you can get more variation than you'd think from a homogenous dataset, though - I've trained a LoRA on myself that can produce photorealistic renderings of me, as well as anime, Disney, Pixar, and Dreamworks styles, all of which actually feel pretty "me"; I don't have any human-produced drawings of myself as a Disney character or renderings of me in a Pixar movie, but the model does a good job of projecting what I'd look like given what it knows about "Disney-style" drawings, plus what I've taught it about what I look like.
cheald 3y ago
For fun, I took two shots of Sonia from the third TOTK trailer (who should be novel and not in the model), trained a LoRA for 400 steps on just them (based on the AbyssOrange anime model), then threw it into an entirely different model (Dreamshaper, which mixes anime and realistic models) with some prompting to get something other than the original. https://imgur.com/a/iPNPojK https://imgur.com/a/iPNPojK I'm not gonna say that it's perfect, but as an interpretation of the character based on exactly two nearly-identical screen grabs trained for a whole 400 steps and some amateur prompting? Not too shabby at all. It hit many of the distinctive points of the character - darker skin, blonde hair with ringlets, the forehead emblem, the tear tattoos under the eyes, even the laurels under the hair. It missed the bell earrings and the secret stone necklace, but given the limited source material, I think it's quite good. With a few more varied screenshots, I'll bet that it could do quite impressively.