6 ms·

Fast-stable-diffusion colabs, +25% speed increase and memory efficient

60 points by DLeychIC 4y ago

xrd 4y ago
I love StableDiffusion, and these projects are so exciting. But. Isn't this the new equivalent of downloading and running a shell script WHICH YOU HAVE NO IDEA WHAT IT DOES? This colab notebook asks for me to give it access to my GDrive. If any of the sd code were modified from the original SD codebase (as I would expect it to be, this is a FORK afterall!) then why couldn't it pull certain anything it wants from my GDrive? Colab does not protect you from this, right? There is no way to sandbox GDrive access, it is all or nothing, right?
- victornomad 4y ago
  Probably if you know how to run the notebook you might also know how to quickly go through the code and see if it might do a potential harm to your hard drive. I dont see a big deal TBH.
  - londons_explore 4y ago
    Buuuuut.... Notice that it appears this repo contains precompiled binaries and no obvious link to the source.... This is literally running some binary blob and giving it access to a big chunk of your digital life. If it were malicious, all it would need to do is steal the token and then upload some illegal images and your whole Google account will be permabanned.
- ricopags 4y ago
  Much like you'd use a sandbox VM to test untrusted code locally, you could use a sandbox google account to work with untrusted Colabs.
- johnsimer 4y ago
  Run it on a burner Gmail account
  - radicaldreamer 4y ago
    In addition to having a google account without any sensitive info, running everything in VMs locally should become a standard practice
    - MacsHeadroom 4y ago
      Colab runs everything remotely unless you point it at a locally installed Jupyter runtime.
  - nomel 4y ago
    How does one make a burner Gmail account these days? Last I tried, a unique phone number was required.
    - FartyMcFarter 4y ago
      Get a burner SIM card ?
    - deepserket 4y ago
      if your goal is to have an empty Drive you can reuse the same number to create a new account. Edit: i just created a new account from my smartphone without a number
- neodypsis 4y ago
  There are Stable Diffusion notebooks out there that don't require GDrive access and run fine in Colab.
- stoplying1 4y ago
  Everything about the SD ecosystem feels like... I'm not sure. It seems like there are tons of different small, uncoordinated efforts, all of which have either no documentation or large plaque install scripts, and then it's all python at the bottom with all of the joys of it's package ecosystem. And then all of these versions where they're so super smart and want access to my GD and an API token to some account of mine? Christ, it would be so, so, so easy to weaponize one of these, add some screenshots and "beat" the original and harvest some Google Drive goodies. In a, I guess I'll say, less frustrating world... there'd be a Nice or Guix expression and it would be relatively trivial to just launch it anywhere Linux and GPU exists.
whywhywhywhy 4y ago
Would be much better if these changes were a branch on the starting repo rather than entire refactors with many changes.
- minimaxir 4y ago
  There are pros and cons to each of these Stable Diffusion hacks (usually time/memory/usability tradeoffs) so it's not correct to merge it back.
  - monkpit 4y ago
    Which is why it would be a branch, no?
  - londons_explore 4y ago
    And in this case it seems like mostly an implementation detail - it could just be a flag to enable, or enabled whenever it detects running on colab.
- eoerl 4y ago
  did you even peek at the link ? There's a PR on diffusers, and it's mentioned on the front page https://github.com/huggingface/diffusers/pull/532#issuecomment-1250134751 https://github.com/huggingface/diffusers/pull/532#issuecomme...
andrewon 4y ago
Can this be run without colab Pro? (High vram)
- ortusdux 4y ago
  The un-optimized version runs in a free notebook, but you hit the vram cap if you try to increase image resolution or generate more than one image at a time.
tgtweak 4y ago
any differences in output or they are identical with the same prompt and seed?
- eoerl 4y ago
  identical outputs, up to float computation shenanigans (not computed in the same order, strictly speaking)
bloaf 4y ago
These guys claim a 50% speedup with a similar approach: https://www.reddit.com/r/StableDiffusion/comments/xmr3ic/speed_up_stable_diffusion_by_50_using_flash/ https://www.reddit.com/r/StableDiffusion/comments/xmr3ic/spe...
- eoerl 4y ago
  yep, same approach but it arrived 3 days later and there's no mention of the [original PR](https://github.com/huggingface/diffusers/pull/532#issuecomment-1250134751 https://github.com/huggingface/diffusers/pull/532#issuecomme...), nice. Else the kernels used in that case -upstream flash attention- are not compatible with all nvidia GPU generations, FYI (xformers' cover a wider range and are generally faster or just pull Flash')

60 points by DLeychIC 4y ago

xrd 4y ago

I love StableDiffusion, and these projects are so exciting. But. Isn't this the new equivalent of downloading and running a shell script WHICH YOU HAVE NO IDEA WHAT IT DOES? This colab notebook asks for me to give it access to my GDrive. If any of the sd code were modified from the original SD codebase (as I would expect it to be, this is a FORK afterall!) then why couldn't it pull certain anything it wants from my GDrive? Colab does not protect you from this, right? There is no way to sandbox GDrive access, it is all or nothing, right?

victornomad 4y ago
Probably if you know how to run the notebook you might also know how to quickly go through the code and see if it might do a potential harm to your hard drive. I dont see a big deal TBH.
- londons_explore 4y ago
  Buuuuut.... Notice that it appears this repo contains precompiled binaries and no obvious link to the source.... This is literally running some binary blob and giving it access to a big chunk of your digital life. If it were malicious, all it would need to do is steal the token and then upload some illegal images and your whole Google account will be permabanned.
ricopags 4y ago
Much like you'd use a sandbox VM to test untrusted code locally, you could use a sandbox google account to work with untrusted Colabs.
johnsimer 4y ago
Run it on a burner Gmail account
- radicaldreamer 4y ago
  In addition to having a google account without any sensitive info, running everything in VMs locally should become a standard practice
  - MacsHeadroom 4y ago
    Colab runs everything remotely unless you point it at a locally installed Jupyter runtime.
- nomel 4y ago
  How does one make a burner Gmail account these days? Last I tried, a unique phone number was required.
  - FartyMcFarter 4y ago
    Get a burner SIM card ?
  - deepserket 4y ago
    if your goal is to have an empty Drive you can reuse the same number to create a new account. Edit: i just created a new account from my smartphone without a number
neodypsis 4y ago
There are Stable Diffusion notebooks out there that don't require GDrive access and run fine in Colab.
stoplying1 4y ago
Everything about the SD ecosystem feels like... I'm not sure. It seems like there are tons of different small, uncoordinated efforts, all of which have either no documentation or large plaque install scripts, and then it's all python at the bottom with all of the joys of it's package ecosystem. And then all of these versions where they're so super smart and want access to my GD and an API token to some account of mine? Christ, it would be so, so, so easy to weaponize one of these, add some screenshots and "beat" the original and harvest some Google Drive goodies. In a, I guess I'll say, less frustrating world... there'd be a Nice or Guix expression and it would be relatively trivial to just launch it anywhere Linux and GPU exists.

whywhywhywhy 4y ago

Would be much better if these changes were a branch on the starting repo rather than entire refactors with many changes.

minimaxir 4y ago
There are pros and cons to each of these Stable Diffusion hacks (usually time/memory/usability tradeoffs) so it's not correct to merge it back.
- monkpit 4y ago
  Which is why it would be a branch, no?
- londons_explore 4y ago
  And in this case it seems like mostly an implementation detail - it could just be a flag to enable, or enabled whenever it detects running on colab.
eoerl 4y ago
did you even peek at the link ? There's a PR on diffusers, and it's mentioned on the front page https://github.com/huggingface/diffusers/pull/532#issuecomment-1250134751 https://github.com/huggingface/diffusers/pull/532#issuecomme...

andrewon 4y ago

Can this be run without colab Pro? (High vram)

ortusdux 4y ago
The un-optimized version runs in a free notebook, but you hit the vram cap if you try to increase image resolution or generate more than one image at a time.

tgtweak 4y ago

any differences in output or they are identical with the same prompt and seed?

eoerl 4y ago
identical outputs, up to float computation shenanigans (not computed in the same order, strictly speaking)

bloaf 4y ago

These guys claim a 50% speedup with a similar approach: https://www.reddit.com/r/StableDiffusion/comments/xmr3ic/speed_up_stable_diffusion_by_50_using_flash/ https://www.reddit.com/r/StableDiffusion/comments/xmr3ic/spe...

eoerl 4y ago
yep, same approach but it arrived 3 days later and there's no mention of the [original PR](https://github.com/huggingface/diffusers/pull/532#issuecomment-1250134751 https://github.com/huggingface/diffusers/pull/532#issuecomme...), nice. Else the kernels used in that case -upstream flash attention- are not compatible with all nvidia GPU generations, FYI (xformers' cover a wider range and are generally faster or just pull Flash')