5 ms·

This is _markedly_ faster than the PyTorch versions I've seen (nothing against the library, just categorizing the implementations). It would be nice to see this

by rcarmo 4y ago

This is _markedly_ faster than the PyTorch versions I've seen (nothing against the library, just categorizing the implementations). It would be nice to see this including the little quality of life additional models (eye fixes, upscaling, etc.), but I suspect the optimizations are transferrable. Either way, getting 3 images for 25 iterations under 10 seconds (quick Colab test, which is where I've taken to comparing these things) is just ridiculously faster.

zone411 4y ago
Which GPU did you test on Colab? Are you comparing with one of the fp16 PyTorch versions? Their test shows little improvement on V100. PyTorch is now quite a bit more popular than Keras in research-type code (except when it comes from Google) so I don't know if these enhancements will get ported. This port was done by people working on Keras which is kind of telling - there isn't a lot of outside interest.
- kgwgk 4y ago
  Should we expect people not working on keras to have the interest and ability to get it to work on keras?
  - zone411 4y ago
    If these people have existing Keras code they want to integrate or they are interested in developing it further in Keras, then it shouldn't require any insider knowledge to create a Keras version of a small but popular open-source project like this. I am very sure we'd get a PyTorch version made by outsiders quickly if Stable Diffusion was originally released in Keras/TF.
    - kgwgk 4y ago
      What is your definition of outsider? We got a Keras version made by Divam Gupta very quickly after Stable Diffusion was released. Is he not an outsider?
      - zone411 4y ago
        From what I can tell this Keras version was just released (the date on the post is Sep. 25) and the first author listed is the creator of Keras. Is this incorrect? I am not familiar with Divam Gupta and I would consider outsiders to be people not paid by Google.
        kgwgk 4y ago
        https://mobile.twitter.com/divamgupta/status/1571234504320208897 https://mobile.twitter.com/divamgupta/status/157123450432020... https://github.com/divamgupta/stable-diffusion-tensorflow https://github.com/divamgupta/stable-diffusion-tensorflow Now they are working together. That may be “telling” to you but I’m not sure why that should cast a negative light on Keras, really.
        zone411 4y ago
        I didn't say that it casts a negative light on Keras. Just on its popularity among outsiders. There are thousands of great libraries out there that are much less popular than Keras or PyTorch. And BTW, JAX is a useful Google-created framework that's growing in popularity among researchers and pushed PyTorch to improve (functorch), so I have nothing against Google projects.
        kgwgk 4y ago
        The reason why we’re having this discussion is that what you call a Keras outsider ported Stable Diffusion to Keras last week. It’s hard to understand how that can say anything negative about the popularity of Keras among outsiders.
        zone411 4y ago
        So why are Keras creators listed as authors on this post and why is it on Keras' official site? Compare this to hundreds of PyTorch SD forks that have been thrown up on GitHub. The OP was wondering whether additional enhancements will also be ported and that's what I responding to. It's simply much less likely that a new paper will get a Keras implementation than a PyTorch implementation.
- _ntka 4y ago
  This is not true, the initial Keras port of the model was done by Divam Gupta who is not affiliated with Keras or Google. He works at Meta. The benchmark in the article uses mixed precision (and equivalent generation settings) for both implementations, it's a fair benchmark. In the latest StackOverflow global developer survey, TensorFlow had 50% more users than PyTorch.
  - zone411 4y ago
    Two Keras creators are listed as authors on this post. If they were not involved, this should be specified. I specifically talked about research and StackOverflow is not in any way representative of what's used. Do you disagree that the majority of neural net research papers now only have PyTorch implementations, not TensorFlow? Also, according to Google Trends, PyTorch is more popular: https://trends.google.com/trends/explore?geo=US&q=pytorch,tensorflow,keras https://trends.google.com/trends/explore?geo=US&q=pytorch,te.... BTW, I would love it if TF made a strong comeback, it's always better to have two big competing frameworks and I have some issues with PyTorch, including with its performance.
  - polygamous_bat 4y ago
    > In the latest StackOverflow global developer survey, TensorFlow had 50% more users than PyTorch. It also doesn't help that PyTorch has its own discussion forum [1] where most pytorch questions end up. [1]: https://discuss.pytorch.org/ https://discuss.pytorch.org/
nextaccountic 4y ago
Is this faster even after applying the optimizations that reduce VRAM usage? (some of which the Keras version seem to lack)

by rcarmo 4y ago

zone411 4y ago

Which GPU did you test on Colab? Are you comparing with one of the fp16 PyTorch versions? Their test shows little improvement on V100. PyTorch is now quite a bit more popular than Keras in research-type code (except when it comes from Google) so I don't know if these enhancements will get ported. This port was done by people working on Keras which is kind of telling - there isn't a lot of outside interest.

kgwgk 4y ago
Should we expect people not working on keras to have the interest and ability to get it to work on keras?
- zone411 4y ago
  If these people have existing Keras code they want to integrate or they are interested in developing it further in Keras, then it shouldn't require any insider knowledge to create a Keras version of a small but popular open-source project like this. I am very sure we'd get a PyTorch version made by outsiders quickly if Stable Diffusion was originally released in Keras/TF.
  - kgwgk 4y ago
    What is your definition of outsider? We got a Keras version made by Divam Gupta very quickly after Stable Diffusion was released. Is he not an outsider?
    - zone411 4y ago
      From what I can tell this Keras version was just released (the date on the post is Sep. 25) and the first author listed is the creator of Keras. Is this incorrect? I am not familiar with Divam Gupta and I would consider outsiders to be people not paid by Google.
      - kgwgk 4y ago
        https://mobile.twitter.com/divamgupta/status/1571234504320208897 https://mobile.twitter.com/divamgupta/status/157123450432020... https://github.com/divamgupta/stable-diffusion-tensorflow https://github.com/divamgupta/stable-diffusion-tensorflow Now they are working together. That may be “telling” to you but I’m not sure why that should cast a negative light on Keras, really.
        zone411 4y ago
        I didn't say that it casts a negative light on Keras. Just on its popularity among outsiders. There are thousands of great libraries out there that are much less popular than Keras or PyTorch. And BTW, JAX is a useful Google-created framework that's growing in popularity among researchers and pushed PyTorch to improve (functorch), so I have nothing against Google projects.
        kgwgk 4y ago
        The reason why we’re having this discussion is that what you call a Keras outsider ported Stable Diffusion to Keras last week. It’s hard to understand how that can say anything negative about the popularity of Keras among outsiders.
        zone411 4y ago
        So why are Keras creators listed as authors on this post and why is it on Keras' official site? Compare this to hundreds of PyTorch SD forks that have been thrown up on GitHub. The OP was wondering whether additional enhancements will also be ported and that's what I responding to. It's simply much less likely that a new paper will get a Keras implementation than a PyTorch implementation.
_ntka 4y ago
This is not true, the initial Keras port of the model was done by Divam Gupta who is not affiliated with Keras or Google. He works at Meta. The benchmark in the article uses mixed precision (and equivalent generation settings) for both implementations, it's a fair benchmark. In the latest StackOverflow global developer survey, TensorFlow had 50% more users than PyTorch.
- zone411 4y ago
  Two Keras creators are listed as authors on this post. If they were not involved, this should be specified. I specifically talked about research and StackOverflow is not in any way representative of what's used. Do you disagree that the majority of neural net research papers now only have PyTorch implementations, not TensorFlow? Also, according to Google Trends, PyTorch is more popular: https://trends.google.com/trends/explore?geo=US&q=pytorch,tensorflow,keras https://trends.google.com/trends/explore?geo=US&q=pytorch,te.... BTW, I would love it if TF made a strong comeback, it's always better to have two big competing frameworks and I have some issues with PyTorch, including with its performance.
- polygamous_bat 4y ago
  > In the latest StackOverflow global developer survey, TensorFlow had 50% more users than PyTorch. It also doesn't help that PyTorch has its own discussion forum [1] where most pytorch questions end up. [1]: https://discuss.pytorch.org/ https://discuss.pytorch.org/

nextaccountic 4y ago

Is this faster even after applying the optimizations that reduce VRAM usage? (some of which the Keras version seem to lack)