4 ms·

The big problem is the variation. If you want a specific, complex scene it's basically impossible to get exactly what you want in my experience. I've also foun

The big problem is the variation. If you want a specific, complex scene it's basically impossible to get exactly what you want in my experience. I've also found that some words and phrases will overpower the rest of the prompt, so it can be hard to get to specific areas of the latent space. I think they're really good for mood board kind of work and exploring ideas, but you'll probably still need an artist to create something that is specifically what you want.

in3d 4y ago
Have you tried ControlNet with Stable Diffusion?
- ZephyrBlu 4y ago
  I have not! Only MJ.
  - in3d 4y ago
    It's a huge step toward getting the image you want, but like others said, MJ is ahead in quality right now.
    - ElFitz 4y ago
      So could one use SD + ControlNet for composition, then Midjourney image to image for the final result?
      - wincy 4y ago
        If img2img works in Midjourney the same way it does in Stable Diffusion, probably not. Img2img doesn’t put “guard rails” to keep everything where you intend it to be and will bleed over and make changes to the image. Civitai.com has some really great models I’ve been using to get some really incredible outputs lately. Generally takes me 1-1/2 hours to get something exactly how I want it, aside from things the model just absolutely doesn’t understand (getting a dragon to breathe fire or try making a woman have tusks like an orc, even with an image showing what it looks like the AI has no concept of what the heck I’m talking about)
drusepth 4y ago
> The big problem is the variation. If you want a specific, complex scene it's basically impossible to get exactly what you want in my experience. The UI/UX of a proper workflow is still very janky (even in user-friendly UIs like auto1111), but you can get very specific, complicated scenes with a mixture of ControlNet + inpainting. > I've also found that some words and phrases will overpower the rest of the prompt, so it can be hard to get to specific areas of the latent space. I've also had this problem (particularly with multiple colored objects, like "blue eyes, brown hair"). Apparently Cutoff (https://github.com/hnmr293/sd-webui-cutoff https://github.com/hnmr293/sd-webui-cutoff) is very good at addressing this "leakage", but I haven't implemented it yet. Right now, things are really good for mood board kind of work, but there's a lot of tech already available for enabling a lot of back-and-forth "work" on images to get them into exactly what you're imagining. There aren't a lot of good UIs for them all yet though; as soon as they get a little more user-friendly, I'd expect another boom in generative AI for artists (this time, with artists being the primary benefactor).
epivosism 4y ago
This is being worked on with a model Deep Floyd supposedly coming out this week. https://www.youtube.com/watch?v=vlxnDNVkWFo https://www.youtube.com/watch?v=vlxnDNVkWFo
cthalupa 4y ago
If working in Stable Diffusion, with ControlNet, Latent Couple, SegmentAnything, and inpainting, you can get incredibly complex scene composition. It's not just typing in a description and getting what you want in one step, but it's still a significantly easier skillset to learn than what would be needed to create the same level of artwork manually. One shot random prompt stuff is better on Midjourney and I'm still subscribed to it, but I would say I do 99% of my playing around inside StableDiffusion these days.
- ElFitz 4y ago
  That, and also simple image to image. Using a simple drawing of single colour shapes as the original image and a good prompt, one can get quite far in terms of composition. The desert dome city still blows my mind every time I think about it.
  - wincy 4y ago
    I drew literal stick figures in GIMP yesterday and went from weird lumpy dragon on a castle drawn by a 3rd grader to an epic terrifying dragon in about ten minutes yesterday. The value of both img2img and then Controlnet cannot be overstated for defining what you want in a scene. Literally my eight year old can do it, given I help her with the prompts.
dragonwriter 4y ago
> If you want a specific, complex scene it's basically impossible to get exactly what you want in my experience. SD with Multidiffusion or another multiple prompts with (potentially overlapping or nested) regions tool, plus some inpainting, can do a lot here. So can SD with ControlNet, to go beyond text prompting.

in3d 4y ago

Have you tried ControlNet with Stable Diffusion?

ZephyrBlu 4y ago
I have not! Only MJ.
- in3d 4y ago
  It's a huge step toward getting the image you want, but like others said, MJ is ahead in quality right now.
  - ElFitz 4y ago
    So could one use SD + ControlNet for composition, then Midjourney image to image for the final result?
    - wincy 4y ago
      If img2img works in Midjourney the same way it does in Stable Diffusion, probably not. Img2img doesn’t put “guard rails” to keep everything where you intend it to be and will bleed over and make changes to the image. Civitai.com has some really great models I’ve been using to get some really incredible outputs lately. Generally takes me 1-1/2 hours to get something exactly how I want it, aside from things the model just absolutely doesn’t understand (getting a dragon to breathe fire or try making a woman have tusks like an orc, even with an image showing what it looks like the AI has no concept of what the heck I’m talking about)

drusepth 4y ago

> The big problem is the variation. If you want a specific, complex scene it's basically impossible to get exactly what you want in my experience. The UI/UX of a proper workflow is still very janky (even in user-friendly UIs like auto1111), but you can get very specific, complicated scenes with a mixture of ControlNet + inpainting. > I've also found that some words and phrases will overpower the rest of the prompt, so it can be hard to get to specific areas of the latent space. I've also had this problem (particularly with multiple colored objects, like "blue eyes, brown hair"). Apparently Cutoff (https://github.com/hnmr293/sd-webui-cutoff https://github.com/hnmr293/sd-webui-cutoff) is very good at addressing this "leakage", but I haven't implemented it yet. Right now, things are really good for mood board kind of work, but there's a lot of tech already available for enabling a lot of back-and-forth "work" on images to get them into exactly what you're imagining. There aren't a lot of good UIs for them all yet though; as soon as they get a little more user-friendly, I'd expect another boom in generative AI for artists (this time, with artists being the primary benefactor).

epivosism 4y ago

This is being worked on with a model Deep Floyd supposedly coming out this week. https://www.youtube.com/watch?v=vlxnDNVkWFo https://www.youtube.com/watch?v=vlxnDNVkWFo

cthalupa 4y ago

If working in Stable Diffusion, with ControlNet, Latent Couple, SegmentAnything, and inpainting, you can get incredibly complex scene composition. It's not just typing in a description and getting what you want in one step, but it's still a significantly easier skillset to learn than what would be needed to create the same level of artwork manually. One shot random prompt stuff is better on Midjourney and I'm still subscribed to it, but I would say I do 99% of my playing around inside StableDiffusion these days.

ElFitz 4y ago
That, and also simple image to image. Using a simple drawing of single colour shapes as the original image and a good prompt, one can get quite far in terms of composition. The desert dome city still blows my mind every time I think about it.
- wincy 4y ago
  I drew literal stick figures in GIMP yesterday and went from weird lumpy dragon on a castle drawn by a 3rd grader to an epic terrifying dragon in about ten minutes yesterday. The value of both img2img and then Controlnet cannot be overstated for defining what you want in a scene. Literally my eight year old can do it, given I help her with the prompts.

dragonwriter 4y ago

> If you want a specific, complex scene it's basically impossible to get exactly what you want in my experience. SD with Multidiffusion or another multiple prompts with (potentially overlapping or nested) regions tool, plus some inpainting, can do a lot here. So can SD with ControlNet, to go beyond text prompting.