3 ms·

Note that this technique and its results are unrelated to the infamous "spiral" ControlNet images a couple months back: https://arstechnica.com/information-tech

Note that this technique and its results are unrelated to the infamous "spiral" ControlNet images a couple months back: https://arstechnica.com/information-technology/2023/09/dreamy-ai-generated-geometric-scenes-mesmerize-social-media-users/ https://arstechnica.com/information-technology/2023/09/dream... Per the code, the technique is based off of DeepFloyd-IF, which is not as easy to run as a Stable Diffusion variant.

SamBam 3y ago
I missed it, what was infamous about it?
- minimaxir 3y ago
  It created a backlash because a) it was too popular with AI people hyping "THIS CHANGES EVERYTHING!" and people were posting low-effort transformations to the point that it got saturated and b) non-AI people were "tricked" into thinking it was a clever trick with real art since ControlNet is not ubiquitous outside the AI-sphere, and they got mad.
  - andybak 3y ago
    I rather liked it and actually didn't get to see as many examples as I wanted to. Is there a good repository anywhere or is it just "wade through twitter"?
    - swyx 3y ago
      not a repository as such but i linked to some good examples in my sept recap https://www.latent.space/p/sep-2023 https://www.latent.space/p/sep-2023 https://github.com/swyxio/ai-notes/blob/main/Monthly%20Notes/Sept%202023%20notes.md https://github.com/swyxio/ai-notes/blob/main/Monthly%20Notes...
  - yreg 3y ago
    It is real art.
Der_Einzige 3y ago
I always thought it was weird that this idea took off with that particular controlnet model. Many other controlnet models when combined with those same images produce excellent and striking results. The ecosystem around Stable Diffusion in general is so massive.
- minimaxir 3y ago
  Other ControlNet adapters either preserve the high-level shape not enough or preserve it too well, IMO. Canny/Depth ControlNet generations are less of an illusion.
ShamelessC 3y ago
> Per the code, the technique is based off of DeepFloyd-IF, which is not as easy to run as a Stable Diffusion variant. I haven't dug in yet, but it _should_ be possible to use their ideas in other diffusion networks? It may be a non-trivial change to the code provided though. Happy to be corrected of course.
- minimaxir 3y ago
  I suspect the trick only works because DeepFloyd-IF operates in pixel space while other diffusion models operate in the latent space. > Our method uses DeepFloyd IF, a pixel-based diffusion model. We do not use Stable Diffusion because latent diffusion models cause artifacts in illusions (see our paper for more details).
deleted 3y ago
[deleted]
tpudlik 3y ago
Did you mean to say it's _related_? The original "spiral" image by Ugleh is explicitly credited in the "Related Links" section.
- minimaxir 3y ago
  It’s a similar topic which is why they credit it but the mechanism is much different.

SamBam 3y ago

I missed it, what was infamous about it?

minimaxir 3y ago
It created a backlash because a) it was too popular with AI people hyping "THIS CHANGES EVERYTHING!" and people were posting low-effort transformations to the point that it got saturated and b) non-AI people were "tricked" into thinking it was a clever trick with real art since ControlNet is not ubiquitous outside the AI-sphere, and they got mad.
- andybak 3y ago
  I rather liked it and actually didn't get to see as many examples as I wanted to. Is there a good repository anywhere or is it just "wade through twitter"?
  - swyx 3y ago
    not a repository as such but i linked to some good examples in my sept recap https://www.latent.space/p/sep-2023 https://www.latent.space/p/sep-2023 https://github.com/swyxio/ai-notes/blob/main/Monthly%20Notes/Sept%202023%20notes.md https://github.com/swyxio/ai-notes/blob/main/Monthly%20Notes...
- yreg 3y ago
  It is real art.

Der_Einzige 3y ago

I always thought it was weird that this idea took off with that particular controlnet model. Many other controlnet models when combined with those same images produce excellent and striking results. The ecosystem around Stable Diffusion in general is so massive.

minimaxir 3y ago
Other ControlNet adapters either preserve the high-level shape not enough or preserve it too well, IMO. Canny/Depth ControlNet generations are less of an illusion.

ShamelessC 3y ago

> Per the code, the technique is based off of DeepFloyd-IF, which is not as easy to run as a Stable Diffusion variant. I haven't dug in yet, but it _should_ be possible to use their ideas in other diffusion networks? It may be a non-trivial change to the code provided though. Happy to be corrected of course.

minimaxir 3y ago
I suspect the trick only works because DeepFloyd-IF operates in pixel space while other diffusion models operate in the latent space. > Our method uses DeepFloyd IF, a pixel-based diffusion model. We do not use Stable Diffusion because latent diffusion models cause artifacts in illusions (see our paper for more details).

deleted 3y ago

[deleted]

tpudlik 3y ago

Did you mean to say it's _related_? The original "spiral" image by Ugleh is explicitly credited in the "Related Links" section.

minimaxir 3y ago
It’s a similar topic which is why they credit it but the mechanism is much different.