20 ms·
Stable Cascade
- obviyus 3y agoBeen using it for a couple of hours and it seems it’s much better at following the prompt. Right away it seems the quality is worse compared to some SDXL models but I’ll reserve judgement until a couple more days of testing. It’s fast too! I would reckon about 2-3x faster than non-turbo SDXL.
- sorenjan 3y agoHow much VRAM does it need? They mention that the largest model uses 1.4 billion parameters more than SDXL, which in turn need a lot of VRAM.
- adventured 3y agoThere was a leak from Japan yesterday, prior to this release, and in that it was suggested 20gb for the largest model. This text was part of the Stability Japan leak (the 20gb VRAM reference was dropped in the release today): "Stages C and B will be released in two different models. Stage C uses parameters of 1B and 3.6B, and Stage B uses parameters of 700M and 1.5B. However, if you want to minimize your hardware needs, you can also use the 1B parameter version. In Stage B, both give great results, but 1.5 billion is better at reconstructing finer details. Thanks to Stable Cascade's modular approach, the expected amount of VRAM required for inference can be kept at around 20GB, but can be reduced even further by using smaller variations (as mentioned earlier, this (which may reduce the final output quality)."
- sorenjan 3y agoThanks. I guess this means that fewer people will be able to use it on their own computer, but the improved efficiency makes it cheaper to run on servers with enough VRAM. Maybe running stage C first, unloading it from VRAM, and then do B and A would make it fit in 12 or even 8 GB, but I wonder if the memory transfers would negate any time saving. Might still be worth it if it produces better images though.
- adventured 3y agoIf it worked I imagine large batching could make it worth the load/unload time cost.
- weebull 3y agoShouldn't be a reason you couldn't do a ton of Layer C work on different images, and then swap in Layer B.
- Filligree 3y agoSequential model offloading isn’t too bad. It adds about a second or less to inference, assuming it still fits in main memory.
- sorenjan 3y agoSometimes I forget how fast modern computers are. PCIe v4 x16 has a transfer speed of 31.5 GB/s, so theoretically it should take less than 100 ms to transfer stage B and A. Maybe it's not so bad after all, it will be interesting to see what happens.
- whywhywhywhy 3y agoIf you're serious about doing image gen locally you should be running a 24GB card anyway because honestly Nvidia's current generation 24GB is the sweet spot price to performance. 3080 ram is laughably the same as the 6 year old 1080Ti and 4080 ram is only slightly more at 16 and costs about 1.5 times the 3090 second hand. Any speed benefits of the 4080 are gonna be worthless the second it has to cycle a model in and out of ram anyway vs the 3090 in image gen.
- weebull 3y ago> because honestly Nvidia's current generation 24GB is the sweet spot price to performance How is the halo product of a range the "sweet spot"? I think nVidia are extremely exposed on this front. The RX 7900XTX is also 24GB and under half the price (In UK at least - £800 vs £1,700 for the 4090). It's difficult to get a performance comparison on compute tasks, but I think it's around 70-80% of the 4090 given what I can find. Even a 3090, if you can find one, is £1,500. The software isn't as stable on AMD hardware, but it does work. I'm running a RX7600 - 8GB myself, and happily doing SDXL. The main problem is that exhausting VRAM causes instability. Exceed it by a lot, and everything is handled fine, but if it's marginal... problems ensue. The AMD engineers are actively making the experience better, and it may not be long before it's a practical alternative. If/When that happens nVidia will need to slash their prices to sell anything in this sphere, which I can't really see themselves doing.
- liuliu 3y agoShould use no more than 6GiB for FP16 models at each stage. The current implementation is not RAM optimized.
- sorenjan 3y agoThe large C model uses 3.6 billion parameters which is 6.7 GiB if each parameter is 16 bits.
- liuliu 3y agoThe large C model have fair bit of parameters tied to text-conditioning, not to the main denoising process. Similar to how we split the network for SDXL Base, I am pretty confident we can split non-trivial amount of parameters to text-conditioning hence during denoising process, loading less than 3.6B parameters.
- brucethemoose2 3y agoWhat's more, they can presumably be swapped in and out like the SDXL base + refiner, right?
- kimoz 3y agoCan one run it on CPU?
- ghurtado 3y agoYou can run any ML model on CPU. The question is the performance
- rwmj 3y agoStable Diffusion on a 16 core AMD CPU takes for me about 2-3 hours to generate an image, just to give you a rough idea of the performance. (On the same AMD's iGPU it takes 2 minutes or so).
- OJFord 3y agoEven older GPUs are worth using then I take it? For example I pulled a (2GB I think, 4 tops) 6870 out of my desktop because it's a beast (in physical size, and power consumption) and I wasn't using it for gaming or anything, figured I'd be fine just with the Intel integrated graphics. But if I wanted to play around with some models locally, it'd be worth putting it back & figuring out how to use it as a secondary card?
- rwmj 3y agoOne counterintuitive advantage of the integrated GPU is it has access to system RAM (instead of using a dedicated and fixed amount of VRAM). That means I'm able to give the iGPU 16 GB of RAM. For me SD takes 8-9 GB of RAM when running. The system RAM is slower than VRAM which is the trade-off here.
- OJFord 3y agoYeah I did wonder about that as I typed, which is why I mentioned the low amount (by modern standards anyway) on the card. OK, thanks!
- mat0 3y agoNo, I don't think so. I think you would need more VRAM to start with.
- vergessenmir 3y agoI'll take prompt adherence over quality any day. The machinery otherwise isn't worth it i.e the controlnets, openpose, depthmaps just to force a particular look or to achieve depth. Th solution becomes bespoke for each generation. Had a test of it and my option is it's an improvement when it comes to following prompts and I do find the images more visually appealing.
- stavros 3y agoCan we use its output as input to SDXL? Presumably it would just fill in the details, and not create whole new images.
- RIMR 3y agoI was thinking that exactly. You could use the same trick as the hires-fix for an adherence-fix.
- emadm 3y agoYeah chain it in comfy to a turbo model for detail
- Filligree 3y agoA turbo model isn't the first thing I'd think of when it comes to finalizing a picture. Have you found one that produces high-quality output?
- dragonwriter 3y agoFor detail, it'd probably be better to use a full model with a small number of steps (something like KSampler Advanced node with 40 total steps, but starting at step 32-ish.) Might even try using the SDXL refiner model for that. Turbo models are decent at low-iteration-decent-results, but not so much at adding fine details to an mostly-done image.
- ttpphd 3y agoThat is a very tiny latent space. Wow!
- yogorenapan 3y agoVery impressive. From what I understand, Stability AI is currently VC funded. It’s bound to burn through tons of money and it’s not clear whether the business model (if any) is sustainable. Perhaps worthy of government funding.
- minimaxir 3y agoStability AI has been burning through tons of money for awhile now, which is the reason newer models like Stable Cascade are not commercially-friendly-licensed open source anymore. > The company is spending significant amounts of money to grow its business. At the time of its deal with Intel, Stability was spending roughly $8 million a month on bills and payroll and earning a fraction of that in revenue, two of the people familiar with the matter said. > It made $1.2 million in revenue in August and was on track to make $3 million this month from software and services, according to a post Mostaque wrote on Monday on X, the platform formerly known as Twitter. The post has since been deleted. https://fortune.com/2023/11/29/stability-ai-sale-intel-ceo-resign/ https://fortune.com/2023/11/29/stability-ai-sale-intel-ceo-r...
- littlestymaar 3y ago> which is the reason newer models like Stable Cascade are not commercially-friendly-licensed open source anymore. The main reason is probably Mid journey and OpenAi using their tech without any kind of contribution back. AI desperately needs a GPL equivalent…
- yogorenapan 3y ago> AI desperately needs a GPL equivalent Why not just the GPL then?
- loudmax 3y agoThe GPL was intended for computer code that gets compiled to a binary form. You can share the binary, but you also have to share the code that the binary is compiled from. Pre-trained model weights might be thought of as analogous to compiled code, and the training data may be analogous to program code, but they're not the same thing. The model weights are shared openly, but the training data used to create these models isn't. This is at least partly because all these models, including OpenAI's, are trained on copyrighted data, so the copyright status of the models themselves is somewhat murky. In the future we may see models that are 100% trained in the open, but foundational models are currently very expensive to train from scratch. Either prices would need to come down, or enthusiasts will need some way to share radically distributed GPU resources.
- jedberg 3y agoI'd say I'm most impressed by the compression. Being able to compress an image 42x is huge for portable devices or bad internet connectivity (or both!).
- incrudible 3y agoThat is 42x spatial compression, but it needs 16 channels instead of 3 for RGB.
- ansk 3y agoFurthermore, each of those 16 channels would typically be mutibyte floats as opposed to single byte RGB channels. (speaking generally, haven't read the paper)
- zamadatix 3y agoEven assuming 32 bit floats (the extra 4 on the end): 4*16*24*24*4 = 147,456 vs (removing the alpha channel as it's unused here) 3*3*1024*1024 = 9,437,184 Or 1/64 raw size, assuming I haven't fucked up the math/understanding somewhere (very possible at the moment).
- incrudible 3y agoIt is actually just 2/4 bytes x 16 latent channels x 24 x 24, but the comparison to raw data needs to be taken with a grain of salt, as there is quite a bit of hallucination involved in reconstruction.
- flgstnd 3y agoa 42x compression is also impressive as it matches the answer to the ultimate question of life, the universe, and everything, maybe there is some deep universal truth within this model.
- seanalltogether 3y agoI have to imagine at this point someone is working toward a fast AI based video codec that comes with a small pretrained model and can operate in a limited memory environment like a tv to offer 8k resolution with low bandwidth.
- yogorenapan 3y agoI see in the commits that the license was changed from MIT to their own custom one: https://github.com/Stability-AI/StableCascade/commit/209a52600f35dfe2a205daef54c0ff4068e86bc7 https://github.com/Stability-AI/StableCascade/commit/209a526... Is it legal to use an older snapshot before the license was changed in accordance with the previous MIT license?
- OJFord 3y agoYes, you can continue to do what you want with that commit^ in accordance with the MIT licence it was released under. Kind of like if you buy an ebook, and then they publish a second edition but only as a hardback - the first edition ebook is still yours to read.
- treesciencebot 3y agoI think the model architecture (training code etc.) itself is still under MIT while the weights (which was the result of training in a huge GPU cluster as well as the dataset they have used [not sure if they publicly talked about it] is under this new license.
- emadm 3y agoCode is MIT, weights are under the NC license for now.
- ed 3y agoIt seems pretty clear the intent was to use a non-commercial license, so it’s probably something that would go to court, if you really wanted to press the issue. Generally courts are more holistic and look at intent, and understand that clerical errors happen. One exception to this is if a business claims it relied on the previous license and invested a bunch of resources as a result. I believe the timing of commits is pretty important— it would be hard to claim your business made a substantial investment on a pre-announcement repo that was only MIT’ed for a few hours.
- RIMR 3y ago
- gorkemyurt 3y agowe have an optimized playground here: https://www.fal.ai/models/stable-cascade https://www.fal.ai/models/stable-cascade
- adventured 3y ago"sign in to run" That's a marketing opportunity being missed, especially given how crowded the space is now. The HN crowd is more likely to run it themselves when presented with signing up just to test out a single generation.
- treesciencebot 3y agoUh, thanks for noticing it! We generally turn it off for popular models so people can see the underlying inference speed and the results but we forgot about it for this one, it should now be auth-less with a stricter rate limit just like other popular models in the gallery.
- RIMR 3y agoI just got rate-limited on my first generation. The message is "You have exceeded the request limit per minute". This was after showing me cli output suggesting that my image was being generated. I guess my zero attempts per minute was too much. You really shouldn't post your product on HN if you aren't prepared for it to work. Reputations are hard to earn, and you're losing people's interest by directing them to a broken product.
- 3y ago
- deleted 3y ago[deleted]
- holoduke 3y agoWow like the compression part. 42 fixed times compression. That is really nice. Slow to unpack on the fly. But the future is waiting.
- GaggiX 3y agoI remember doing some random experiments with these two researchers to find the best way to condition the stage B on the latent, my very fancy cross-attn with relative 2D positional embeddings didn't work as well as just concatenating the channels of the input with the nearest upsample of the latent, so I just gave up ahah. This model used to be known as Würstchen v3.
- joshelgar 3y agoWhy are they benchmarking it with 20+10 steps vs. 50 steps for the other models?
- liuliu 3y agoprior generations usually take fewer steps than vanilla SDXL to reach the same quality. But yeah, the inference speed improvement is mediocre (until I take a look at exactly what computation performed to have more informed opinion on whether it is implementation issue or model issue). The prompt alignment should be better though. It looks like the model have more parameters to work with text conditioning.
- treesciencebot 3y agoin my observation, it yields amazing perf at higher batch sizes (4 or better 8). i assume it is due to memory bandwith and the constrained latent space helping.
- Filligree 3y agoHowever, the outputs are so similar that I barely feel a need for more than 1. 2 is plenty.
- GaggiX 3y agoI think that this model used consistency loss during training so that it can yield better results with less steps.
- weebull 3y ago...because they feel that at 20+10 it achieves a superior output than at 50 steps for SDXL. They also benchmark it against 1 step for SDXL-Turbo.
- rosmery 3y ago[flagged]
- t3estabc 3y ago[dead]
- gajnadsgjoas 3y agoWhere can I run it if I don't have a GPU? Colab didn't work
- detolly 3y agorunpod, kaggle, lambda labs, or pretty much any other server provider that gives you one or more gpus.
- t3estabc 3y ago[dead]
- k2enemy 3y agoI haven't been following the image generation space since the initial excitement around stable diffusion. Is there an easy to use interface for the new models coming out? I remember setting up the python env for stable diffusion, but then shortly after there were a host of nice GUIs. Are there some popular GUIs that can be used to try out newer models? Similarly, what's the best GUI for some of the older models? Preferably for macos.
- thot_experiment 3y agoAuto1111 and Comfy both get updated pretty quickly to support most of the new models coming out. I expect they'll both support this soon.
- stereobit 3y agoCheck out invoke.com
- sophrocyne 3y agoThanks for calling us out - I'm one of the maintainers. Not entirely sure we'll be in the Stable Cascade race quite yet. Since Auto/Comfy aren't really built for businesses, they'll get it incorporated sooner vs later. Invoke's main focus is building open-source tools for the pros using this for work that are getting disrupted, and non-commercial licenses don't really help the ones that are trying to follow the letter of the license. Theoretically, since we're just a deployment solution, it might come up with our larger customers who want us to run something they license from Stability, but we've had zero interest on any of the closed-license stuff so far.
- yokto 3y agofal.ai is nice and fast: https://news.ycombinator.com/item?id=39360800 https://news.ycombinator.com/item?id=39360800 Both in performance and for how quickly they integrate new models apparently: they already support Stable Cascade.
- brucethemoose2 3y ago
- cybereporter 3y agoWill this get integrated into Stable Diffusion Web UI?
- ttul 3y agoSurely within days. ComfyUI’s maintainer said he is readying the node for release perhaps by this weekend. The Stable Cascade model is otherwise known as Würschten v3 and has been floating around the open source generative image space since fall.
- dragonwriter 3y agoThird-party (using diffusers) node for ComfyUI is already available for those who can't wait for native integration. https://github.com/kijai/ComfyUI-DiffusersStableCascade https://github.com/kijai/ComfyUI-DiffusersStableCascade
- hncomb 3y agoIs there any way this can be used to generate multiple images of the same model? e.g. a car model rotated around (but all images are of the same generated car)
- matroid 3y agoSomeone with resources will have to train Zero123 [1] with this backbone. [1] https://zero123.cs.columbia.edu/ https://zero123.cs.columbia.edu/
- emadm 3y agoHeh https://stability.ai/news/stable-zero123-3d-generation https://stability.ai/news/stable-zero123-3d-generation Better coming
- refulgentis 3y agoYes, input image => embedding => N images, and if you're thinking 3D perspectives for rendering, you'd ControlNet the N. ref.: "The model can also understand image embeddings, which makes it possible to generate variations of a given image (left). There was no prompt given here."
- taejavu 3y agoThe model looks different in each of those variations though. Which seems to be intentional, but the post you're responding to is asking whether it's possible to keep the model exactly the same in each render, varying only by perspective.
- LuciBb 3y ago[dead]
- hansonpeter 3y ago[dead]
- ionwake 3y agoDoes anyone have a link to a demo online?
- pxoe 3y agothe way it's written about in Image Reconstruction section like it is just an image compression thing...is kind of interesting. for that stuff and its presented use there to be very much about storing images and reconstructing them. when "it doesn't actually store original images" and "it can't actually give out original images" are points that get used so often in arguments as a defense for image generators. so it is just a multi-image compression file format, just a very efficient one. sure, it's "redrawing"/"rendering" its output and makes things look kinda fuzzy, but any other compressed image format does that as well. what was all that 'well it doesn't do those things' nonsense about then? clearly it can do that.
- wongarsu 3y agoIn a way it's just an algorithm than can compress either text or an image. The neat trick is that if you compress the text "brown bear hitting Vladimir Putin" and then decompress it as an image, you get an image of a bear hitting Vladimir Putin. This principle is the idea behind all Stable Diffusion models, this one "just" achieved a much better compression ratio
- pxoe 3y agowell yeah. but it's not so much about what it actually does, but how they talk about it. maybe (probably) i missed them putting out something that's described like that before, but it's just the open admission in demonstration of it. i guess they're getting more brazen, given than they're not really getting punished for what they're doing, be it piracy or infringement or whatever.
- Filligree 3y agoThe model works on compressed data. That's all it is. Sure, it could output a picture from its training set on decompression, but only if you feed that same picture into the compressor. In which case what are you doing, exactly? Normally you feed it a text prompt instead, which won't compress to the same thing.
- mise_en_place 3y agoWas anyone able to get this running on Colab? I got as far as loading extras in text-to-inference, but it was complaining about a dependency.
- SECourses 3y agoIt is pretty good I shared a comparison on medium https://medium.com/@furkangozukara/stable-cascade-prompt-following-is-amazing-this-model-has-huge-potential-high-resolutions-uses-29088c4ad171 https://medium.com/@furkangozukara/stable-cascade-prompt-fol... My Gradio APP even works amazing on 8 GB gpu with CPU offloading
- lqcfcjx 3y agoI'm very impressed by the recent AI progress on making models smaller and more efficient. I just have the feeling that every week there's something big on this space (like what we saw previously from ollama, llava, mixtral...). Apparently the space for on-device models are not fully discovered yet. Very excited to see future products on that direction.
- dragonwriter 3y ago> I'm very impressed by the recent AI progress on making models smaller and more efficient. That's an odd comment to place in a thread about an image generation model that is bigger than SDXL. Yes, it works in a smaller latent space, yes its faster in the hardware configuration they've used, but its not smaller.
- lqcfcjx 3y agomy bad, I misread the post and got the impression that it's a way smaller model. Thanks for correcting me.
- skybrian 3y agoLike every other image generator I've tried, it can't do a piano keyboard [1]. I expect that some different approach is needed to be able to count the black keys groups. [1] https://fal.ai/models/stable-cascade?share=13d35b76-d32f-45ca-be08-25d2fcb7349d https://fal.ai/models/stable-cascade?share=13d35b76-d32f-45c...
- Agraillo 3y agoI think it's more than this. In my case in most of images I made about basketball there were more than one ball. I'm not an expert, but some fundamental constrains of the human (cultural) life (like all piano keys are the same, there's only one ball in a game) are not grasped by the training or grasped partially
- Doxin 3y agoThese image generators can't count. Go try variations on "one ball", "two balls", etc, and you can see in what sort of ways it fucks it up.
- GaggiX 3y agoAs with human hands, coherency is fixed by scaling the model and the training.
- sanroot99 3y agoWhat is the system requirements needed to run this, particularly how much vram it would take?
- instagraham 3y agoWill this work on AMD? Found no mention of support. Kinda an important feature for such a project, as AMD users running Stable Diffusion will be suffering diminished performance.
- drbscl 3y agoApparently yes https://news.ycombinator.com/item?id=39360106#39360497 https://news.ycombinator.com/item?id=39360106#39360497
- instagraham 3y agoI believe those are CPU specific, it disregards my AMD GPU. It's too slow a proces. For what it's work, a sort of hybrid CPU GPU approach is giving me workable results with automatic111, about 30-50s to generate a 512x512 image ataround 1-2 it/s. Not enough for me to want to use local-run txt2img for daily use (why, when Bing/Poe exists and give you high quality image gen for free in seconds), but enough that when I want some granular control or don't want to give my data to a company (i like making AI portraits of myself and friends), I can run locally. If I could improve performance, I'd love to get into video production
- xkgt 3y agoThis model is built upon the Würstchen architecture. Here is a very good explanation of how this model works by one of its authors. https://www.youtube.com/watch?v=ogJsCPqgFMk https://www.youtube.com/watch?v=ogJsCPqgFMk
- lordswork 3y agoGreat video! And here's a summary of the video :) Gemini Advanced> Summarize this video: https://www.youtube.com/watch?v=ogJsCPqgFMk This video is about a new method for training text-to-image diffusion models called Würstchen. The method is significantly more efficient than previous methods, such as Stable Diffusion 1.4, and can achieve similar results with 16 times less training time and compute. The key to Würstchen's efficiency is its use of a two-stage compression process. The first stage uses a VQ-VAE to compress images into a latent space that is 4 times smaller than the latent space used by Stable Diffusion. The second stage uses a diffusion model to further compress the latent space by another factor of 10. This results in a total compression ratio of 40, which is significantly higher than the compression ratio of 8 used by Stable Diffusion. The compressed latent space allows the text-to-image diffusion model in Würstchen to be much smaller and faster to train than the model in Stable Diffusion. This makes it possible to train Würstchen on a single GPU in just 24,000 GPU hours, while Stable Diffusion 1.4 requires 150,000 GPU hours. Despite its efficiency, Würstchen is able to generate images that are of comparable quality to those generated by Stable Diffusion. In some cases, Würstchen can even generate images that are of higher quality, such as images with higher resolutions or images that contain more detail. Overall, Würstchen is a significant advance in the field of text-to-image generation. It makes it possible to train text-to-image models that are more efficient and affordable than ever before. This could lead to a wider range of applications for text-to-image generation, such as creating images for marketing materials, generating illustrations for books, or even creating personalized avatars.
- nialv7 3y agoCan Stable Cascade be used for image compression? 1024x1024 to 24x24 is crazy.
- anonuser1234 3y agoThat's definitely not lossless compression