4 ms·
Hallucinations. Larger, better trained, multimodal models are needed. All current image generation models are too small to be usable in serious settings. Until
by orbital-decay 3y ago
Hallucinations. Larger, better trained, multimodal models are needed. All current image generation models are too small to be usable in serious settings.
Until you have all that, you can minimize the hallucinations in the result. I have some tips to share (not for the music though).
-1. You still have to know how to draw, otherwise you won't have the eye to spot the "turds" nor skills to fix them. It will still be faster than drawing absolutely everything by hand.
0. Don't use Midjourney for anything complex. It hallucinates well and gives good first impression, but it's simply not suited for anything that has complex composition and lots of meaningful detail that need to be controlled like the examples in the article.
1. Finetune your model on its own output and your own work.
2. Use higher order guidance, not just text. Besides the fact that current models don't understand the natural language well (let alone your intent), text is inherently unsuited for explaining artistic intent. Use controlnets on your sketches, latent noise spatial composition, and other similar tricks - they are just better at that, faster, and more precise.
3. Don't generate everything in one piece, photobash. Make complex compositions out of simpler objects. The model can't keep the track of too much stuff. Use automated workflows to save time (otherwise what's the point?). Check out ComfyUI for a powerful node-based editor - create a pipeline once and use it many times. Use plugins for the common software like Photoshop, Krita, Blender, Houdini etc.
4. Fix your errors manually. If your work is complex, it will be faster.
>But I feel like something that gets lost in the discussion of AI-generated artwork is the turds.
No, it's not. It's the major obstacle in the current models, and gets a lot of attention.