5 ms·

No model, no code. About as exciting as Imagen-video, which was released eons ago.

ipsum2 3y ago
The approach is different, it can use pretrained models, i.e. stable diffusion, which is a pretty exciting research development. This means that it only requires 'fine-tuning' existing models to get this result.
- 152334H 3y ago
  I agree with that, but it's hard for me to get excited with the knowledge that it'll almost certainly be discarded and forgotten. I've seen too many papers that looked interesting from a theoretical perspective, but were simply never brought to the public because of the barrier of dev+training. In this case, you need someone that can implement the method as described (hard!), and then you need someone with a setup better than a rented 8xA100 (expensive and not available on many cloud providers) to actually reproduce the model.
  - ipsum2 3y ago
    To put it in context, in almost all areas of research (physics, biology, chemistry, electronics, etc), running experiments is expensive. ML is in the category that there can still be advances done by amateurs at home. I don't think it's worth writing off everything that requires more resources than a hobbyist.
    - alwayslikethis 3y ago
      Yeah. You can rent 8xA100s, it's a steep price but you can. It's much harder for a hobbyist to rent electron microscopes or NMR machines.
p-e-w 3y ago
I doubt you'll ever again see a model being publicly released for something as capable as this. I can't say I fully understand the mechanisms by which they achieve that, but it's clear that the powers that be have decided that the public cannot be trusted with powerful AI models. Stable Diffusion and LLaMA were mistakes that won't be repeated anytime soon.
- valine 3y ago
  AI models with obvious, broad, real world applications always seem get reproduced in public. Nvidia’s result is obviously great, but it’s still a long way from being useful. It reminds me of image generation models maybe 4 years ago. We need a killer application for video generation models first, then I’m sure someone will throw a $100k at training an open source version.
  - ipsum2 3y ago
    The tragedy of the commons is at play here. We could get amazing ML models rivaling the best if people interested could pool together money for a $100k or $1 million training run. But there doesn't seem to be the willingness, so we humbly rely on the benevolence of companies like Stability and Meta to release models to the public.
    - nomel 3y ago
      As someone not very familiar with the field, what is wrong with this comment? Is the scale off? Kickstarter is into the tens of millions these days [1]. I would assume some number of millions might be possible, if the right names were behind it. [1] https://www.kickstarter.com/projects/dragonsteel/surprise-four-secret-novels-by-brandon-sanderson https://www.kickstarter.com/projects/dragonsteel/surprise-fo...
      - ipsum2 3y ago
        Kickstarters like the one you linked don't suffer from the tragedy of the commons because people are essentially just pre-paying for a product. With funding an open source ML model, there's little incentive to not be a free-rider.
astrange 3y ago
Emad says they will release it. https://twitter.com/emostaque/status/1648546670659923970 https://twitter.com/emostaque/status/1648546670659923970
- ShamelessC 3y ago
  Ugh, that tweet is so confusing. The researchers did work for Stability - this work was done for NVIDIA. It's entirely unclear if the researchers are even still associated with stability.ai but Emad sure does imply that's the case. > (Team working on our variant, will be done when it's done and they are happy with it). I _think_ he's saying that _his_ team is working on a similar model - and that they will release _that_ model "when it's done" (and not to to expect that to happen any time soon). Just super vague, bordering on taking credit for work that NVIDIA did. Seems like he typed it out on his phone and/or is Elon-levels of lazy about tweets.
  - tmabraham 3y ago
    Three of the authors of this paper work for Stability AI currently. One of them even said: "Unfortunately we cannot release the weights. That's why I joined @StabilityAI to work on OS video models" - https://twitter.com/andi_blatt/status/1648598423526932483 https://twitter.com/andi_blatt/status/1648598423526932483
    - ShamelessC 3y ago
      Thanks for the clarification!

The approach is different, it can use pretrained models, i.e. stable diffusion, which is a pretty exciting research development. This means that it only requires 'fine-tuning' existing models to get this result.

152334H 3y ago
I agree with that, but it's hard for me to get excited with the knowledge that it'll almost certainly be discarded and forgotten. I've seen too many papers that looked interesting from a theoretical perspective, but were simply never brought to the public because of the barrier of dev+training. In this case, you need someone that can implement the method as described (hard!), and then you need someone with a setup better than a rented 8xA100 (expensive and not available on many cloud providers) to actually reproduce the model.
- ipsum2 3y ago
  To put it in context, in almost all areas of research (physics, biology, chemistry, electronics, etc), running experiments is expensive. ML is in the category that there can still be advances done by amateurs at home. I don't think it's worth writing off everything that requires more resources than a hobbyist.
  - alwayslikethis 3y ago
    Yeah. You can rent 8xA100s, it's a steep price but you can. It's much harder for a hobbyist to rent electron microscopes or NMR machines.

p-e-w 3y ago

I doubt you'll ever again see a model being publicly released for something as capable as this. I can't say I fully understand the mechanisms by which they achieve that, but it's clear that the powers that be have decided that the public cannot be trusted with powerful AI models. Stable Diffusion and LLaMA were mistakes that won't be repeated anytime soon.

valine 3y ago
AI models with obvious, broad, real world applications always seem get reproduced in public. Nvidia’s result is obviously great, but it’s still a long way from being useful. It reminds me of image generation models maybe 4 years ago. We need a killer application for video generation models first, then I’m sure someone will throw a $100k at training an open source version.
- ipsum2 3y ago
  The tragedy of the commons is at play here. We could get amazing ML models rivaling the best if people interested could pool together money for a $100k or $1 million training run. But there doesn't seem to be the willingness, so we humbly rely on the benevolence of companies like Stability and Meta to release models to the public.
  - nomel 3y ago
    As someone not very familiar with the field, what is wrong with this comment? Is the scale off? Kickstarter is into the tens of millions these days [1]. I would assume some number of millions might be possible, if the right names were behind it. [1] https://www.kickstarter.com/projects/dragonsteel/surprise-four-secret-novels-by-brandon-sanderson https://www.kickstarter.com/projects/dragonsteel/surprise-fo...
    - ipsum2 3y ago
      Kickstarters like the one you linked don't suffer from the tragedy of the commons because people are essentially just pre-paying for a product. With funding an open source ML model, there's little incentive to not be a free-rider.

astrange 3y ago

Emad says they will release it. https://twitter.com/emostaque/status/1648546670659923970 https://twitter.com/emostaque/status/1648546670659923970

ShamelessC 3y ago
Ugh, that tweet is so confusing. The researchers did work for Stability - this work was done for NVIDIA. It's entirely unclear if the researchers are even still associated with stability.ai but Emad sure does imply that's the case. > (Team working on our variant, will be done when it's done and they are happy with it). I _think_ he's saying that _his_ team is working on a similar model - and that they will release _that_ model "when it's done" (and not to to expect that to happen any time soon). Just super vague, bordering on taking credit for work that NVIDIA did. Seems like he typed it out on his phone and/or is Elon-levels of lazy about tweets.
- tmabraham 3y ago
  Three of the authors of this paper work for Stability AI currently. One of them even said: "Unfortunately we cannot release the weights. That's why I joined @StabilityAI to work on OS video models" - https://twitter.com/andi_blatt/status/1648598423526932483 https://twitter.com/andi_blatt/status/1648598423526932483
  - ShamelessC 3y ago
    Thanks for the clarification!