3 ms·

GitHub: https://github.com/deep-floyd/IF https://github.com/deep-floyd/IF Colab Notebook for running the model based on the diffusers library: https://colab.re

GitHub: https://github.com/deep-floyd/IF https://github.com/deep-floyd/IF Colab Notebook for running the model based on the diffusers library: https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/deepfloyd_if_free_tier_google_colab.ipynb https://colab.research.google.com/github/huggingface/noteboo... Hugging Face Space for testing the model: https://huggingface.co/spaces/DeepFloyd/IF https://huggingface.co/spaces/DeepFloyd/IF Note that the model is substantially more compute-intensive than Stable Diffusion, so it may be slower even though that space is running on an A100.

mkaic 4y ago
HF also wrote a blog post on how you can mess around with the model in a python notebook using their excellent Diffusers library: https://huggingface.co/blog/if https://huggingface.co/blog/if
- minimaxir 4y ago
  I knew the model would have difficulty fitting into a 16GB VRAM GPU, but "you need to load and unload parts of the model pipeline to/from the GPU" is not a workaround I expected. At that point it's probably better to write a guide on how to set up a VM with a A100 easily instead of trying to fit it into a Colab GPU.
  - mirekrusin 4y ago
    What about people having RTX 4090 with 24GB or even dual? Does it run on it?
deleted 4y ago
[deleted]
bwesz 4y ago
I tried the HF Space and it generates images of 64x64 resolution, which are basically useless.
- ollin 4y ago
  It generates 64x64 for the first stage, but there's a button to upscale your favorite 64x64 image to usable resolution.
- GaggiX 4y ago
  There is a big "Upscale" button.
ShamelessC 4y ago
> Note that the model is substantially more compute-intensive than Stable Diffusion, so it may be slower even though that space is running on an A100. Any more specifics on this? Sampling has gotten better too. My primary concern is the amount of memory necessary for a generation (batch size = 1, or I guess "2" using classifier free guidance). Similar to OpenAI's cascaded diffusion models and GLIDE, you can presumably run the models in sequence, unloading earlier models from memory to make room for the 2nd and 3rd stage models. Right now, I really only need 256px resolution. So, will I be able to fit the first stage (64px) model in memory on its own with my 12 GB RTX 3060? What about the 2nd stage (64px -> 256px)? 3rd stage?
- dragonwriter 4y ago
  They’ve said the whole pipeline (using the large models in each step with a choice, IIRC) can be run sequenced with 14GB VRAM.

mkaic 4y ago

HF also wrote a blog post on how you can mess around with the model in a python notebook using their excellent Diffusers library: https://huggingface.co/blog/if https://huggingface.co/blog/if

minimaxir 4y ago
I knew the model would have difficulty fitting into a 16GB VRAM GPU, but "you need to load and unload parts of the model pipeline to/from the GPU" is not a workaround I expected. At that point it's probably better to write a guide on how to set up a VM with a A100 easily instead of trying to fit it into a Colab GPU.
- mirekrusin 4y ago
  What about people having RTX 4090 with 24GB or even dual? Does it run on it?

deleted 4y ago

[deleted]

bwesz 4y ago

I tried the HF Space and it generates images of 64x64 resolution, which are basically useless.

ollin 4y ago
It generates 64x64 for the first stage, but there's a button to upscale your favorite 64x64 image to usable resolution.
GaggiX 4y ago
There is a big "Upscale" button.

ShamelessC 4y ago

> Note that the model is substantially more compute-intensive than Stable Diffusion, so it may be slower even though that space is running on an A100. Any more specifics on this? Sampling has gotten better too. My primary concern is the amount of memory necessary for a generation (batch size = 1, or I guess "2" using classifier free guidance). Similar to OpenAI's cascaded diffusion models and GLIDE, you can presumably run the models in sequence, unloading earlier models from memory to make room for the 2nd and 3rd stage models. Right now, I really only need 256px resolution. So, will I be able to fit the first stage (64px) model in memory on its own with my 12 GB RTX 3060? What about the 2nd stage (64px -> 256px)? 3rd stage?

dragonwriter 4y ago
They’ve said the whole pipeline (using the large models in each step with a choice, IIRC) can be run sequenced with 14GB VRAM.