3 ms·
If this is the same old CLIP model, I tried making anime with CLIP+GANs (usual prompt "chitanda from the anime hyouka") and it got the hair color right but noth
by astrange 5y ago
If this is the same old CLIP model, I tried making anime with CLIP+GANs (usual prompt "chitanda from the anime hyouka") and it got the hair color right but nothing else - seemed to have first page of Google Image Search level knowledge.
There have to be a lot of limits once you get far enough out of its training data. Even if you could guide it by image prompts, it has to know what that image is meant to represent.
Also what most people mean by "anime style" isn't anime style, of course - they actually want one of Avatar-type cartoons, advertising key visuals, game characters, or pixiv/DeviantArt fanart. None of that is drawn the same way as TV anime.
- ShamelessC 5y agoThey trained a new noise-aware one on the same dataset, I believe. The CLIP is frozen though in the second stage where they train on a cleaner dataset for the image generation via diffusion conditioned on a CLIP image embed. They get the image embed from another model that can predict image embeds from text embeds to encourage generalizing when sampling. It does suffer from some of the same issues with binding attributes and counting stuff, as far as I can tell. But that's not necessarily related to its ability to compose concepts.