4 ms·

Are y'all running the diffusion in PyTorch eager mode? AITemplate, stable-fast, or even a torch.compile could get that down to 60ms, I bet. Though I'm not sure

• erwannmillon 3y ago
  LCMs are actually exactly SD architecture. LCM is initialized from a regular SD unet and finetuned on a new objective. We are already compiling to get to these times. A lot of other people getting sub-100ms times are using fewer inference steps than we do, at a quality tradeoff.
  • brucethemoose2 3y ago
    > already compiling Hmm, well if you mean torch.compile, y'all should still check out stable-fast, which is claiming ~16ms/iter on a 4090, twice that of torch.compile: https://github.com/chengzeyi/stable-fast#rtx-4090-512x512-batch-size-1-fp16-tcmalloc-enabled https://github.com/chengzeyi/stable-fast#rtx-4090-512x512-ba...
• dvrp 3y ago
  some people even claim they get ~50ms on (powerful) consumer hardware (e.g. RTX 4090). we still have not been able to replicate those results; also because we want to do it in a distributed way
  • brucethemoose2 3y ago
    They might be "cheating" with methods that significantly degrade quality.