3 ms·

Why couldn't it be applied to SD?

• m00x 3y ago
  It looks like it's already at thing https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/f865d3e11647dfd6c7b2cdf90dde24680e58acd8/modules/cmd_args.py#L54 https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob...
  • zaptrem 3y ago
    That's Flash Attention 1, but yeah there's nothing stopping someone from submitting a PR.
  • hunkins 3y ago
    This enables V1. V2 is still yet to be integrated into xformers. The team replied saying it should happen this week. See the relevant Github issue here: https://github.com/facebookresearch/xformers/issues/795 https://github.com/facebookresearch/xformers/issues/795
• mcyc 3y ago
  SD1 uses a head dimension that is not supported by the old version of Flash Attention but is supported by this version.