5 ms·

I find Stable Diffusion better overall, but it has downsides. Stable Diffusion tends to be more creative than DALL-E, but does a lousy job of following directio

I find Stable Diffusion better overall, but it has downsides. Stable Diffusion tends to be more creative than DALL-E, but does a lousy job of following directions, especially complex ones. DALL-E is good if I know what I want specifically. I can think of ways to fix Stable Diffusion since it's open-source. I think I could bridge the gaps as I see them in about a weekend of hacking. I'm not sure when I'll get that weekend. (Footnote: What I want to do is not something I can explain without a technical blog-post-length document or a zoom call; it's about the same level of complexity as the other major SD hacks we've seen)

cube2222 4y ago
Setting a high cfg parameter, like 13, drastically helps with the prompt following. That said, for me, I agree that dalle does much better pencil sketches.
- astrange 4y ago
  Raising the cfg ("classifier-free guidance") scale is essential for following the prompt, but if you raise it too high the image gets weird and saturated. According to Google's Imagen paper this is literally because the pixels get multiplied by the cfg scale and start clipping; they have a technique called dynamic thresholding that replaces it. Not sure if SD uses this, but I saw Emad hinting they were training an Imagen model…
Miraste 4y ago
Something like prompt weighting? I've seen implementations of that floating around.