5 ms·

There was a leak from Japan yesterday, prior to this release, and in that it was suggested 20gb for the largest model. This text was part of the Stability Japa

There was a leak from Japan yesterday, prior to this release, and in that it was suggested 20gb for the largest model. This text was part of the Stability Japan leak (the 20gb VRAM reference was dropped in the release today): "Stages C and B will be released in two different models. Stage C uses parameters of 1B and 3.6B, and Stage B uses parameters of 700M and 1.5B. However, if you want to minimize your hardware needs, you can also use the 1B parameter version. In Stage B, both give great results, but 1.5 billion is better at reconstructing finer details. Thanks to Stable Cascade's modular approach, the expected amount of VRAM required for inference can be kept at around 20GB, but can be reduced even further by using smaller variations (as mentioned earlier, this (which may reduce the final output quality)."

sorenjan 3y ago
Thanks. I guess this means that fewer people will be able to use it on their own computer, but the improved efficiency makes it cheaper to run on servers with enough VRAM. Maybe running stage C first, unloading it from VRAM, and then do B and A would make it fit in 12 or even 8 GB, but I wonder if the memory transfers would negate any time saving. Might still be worth it if it produces better images though.
- adventured 3y ago
  If it worked I imagine large batching could make it worth the load/unload time cost.
  - weebull 3y ago
    Shouldn't be a reason you couldn't do a ton of Layer C work on different images, and then swap in Layer B.
- Filligree 3y ago
  Sequential model offloading isn’t too bad. It adds about a second or less to inference, assuming it still fits in main memory.
  - sorenjan 3y ago
    Sometimes I forget how fast modern computers are. PCIe v4 x16 has a transfer speed of 31.5 GB/s, so theoretically it should take less than 100 ms to transfer stage B and A. Maybe it's not so bad after all, it will be interesting to see what happens.
- whywhywhywhy 3y ago
  If you're serious about doing image gen locally you should be running a 24GB card anyway because honestly Nvidia's current generation 24GB is the sweet spot price to performance. 3080 ram is laughably the same as the 6 year old 1080Ti and 4080 ram is only slightly more at 16 and costs about 1.5 times the 3090 second hand. Any speed benefits of the 4080 are gonna be worthless the second it has to cycle a model in and out of ram anyway vs the 3090 in image gen.
  - weebull 3y ago
    > because honestly Nvidia's current generation 24GB is the sweet spot price to performance How is the halo product of a range the "sweet spot"? I think nVidia are extremely exposed on this front. The RX 7900XTX is also 24GB and under half the price (In UK at least - £800 vs £1,700 for the 4090). It's difficult to get a performance comparison on compute tasks, but I think it's around 70-80% of the 4090 given what I can find. Even a 3090, if you can find one, is £1,500. The software isn't as stable on AMD hardware, but it does work. I'm running a RX7600 - 8GB myself, and happily doing SDXL. The main problem is that exhausting VRAM causes instability. Exceed it by a lot, and everything is handled fine, but if it's marginal... problems ensue. The AMD engineers are actively making the experience better, and it may not be long before it's a practical alternative. If/When that happens nVidia will need to slash their prices to sell anything in this sphere, which I can't really see themselves doing.
    - zargon 3y ago
      > If/When that happens nVidia will need to slash their prices to sell anything in this sphere It's just as likely that AMD will raise prices to compensate.
      - weebull 3y ago
        You think they're going to say "Hey, compute became competitive but nothing else changed performance therefore... PRICE HIKE!"? They don't have the reputation to burn in this domain for that IMHO. Granted you could see a supply/demand related increase from retailers if demand spiked, but that's the retailers capitalising.

sorenjan 3y ago

Thanks. I guess this means that fewer people will be able to use it on their own computer, but the improved efficiency makes it cheaper to run on servers with enough VRAM. Maybe running stage C first, unloading it from VRAM, and then do B and A would make it fit in 12 or even 8 GB, but I wonder if the memory transfers would negate any time saving. Might still be worth it if it produces better images though.

adventured 3y ago
If it worked I imagine large batching could make it worth the load/unload time cost.
- weebull 3y ago
  Shouldn't be a reason you couldn't do a ton of Layer C work on different images, and then swap in Layer B.
Filligree 3y ago
Sequential model offloading isn’t too bad. It adds about a second or less to inference, assuming it still fits in main memory.
- sorenjan 3y ago
  Sometimes I forget how fast modern computers are. PCIe v4 x16 has a transfer speed of 31.5 GB/s, so theoretically it should take less than 100 ms to transfer stage B and A. Maybe it's not so bad after all, it will be interesting to see what happens.
whywhywhywhy 3y ago
If you're serious about doing image gen locally you should be running a 24GB card anyway because honestly Nvidia's current generation 24GB is the sweet spot price to performance. 3080 ram is laughably the same as the 6 year old 1080Ti and 4080 ram is only slightly more at 16 and costs about 1.5 times the 3090 second hand. Any speed benefits of the 4080 are gonna be worthless the second it has to cycle a model in and out of ram anyway vs the 3090 in image gen.
- weebull 3y ago
  > because honestly Nvidia's current generation 24GB is the sweet spot price to performance How is the halo product of a range the "sweet spot"? I think nVidia are extremely exposed on this front. The RX 7900XTX is also 24GB and under half the price (In UK at least - £800 vs £1,700 for the 4090). It's difficult to get a performance comparison on compute tasks, but I think it's around 70-80% of the 4090 given what I can find. Even a 3090, if you can find one, is £1,500. The software isn't as stable on AMD hardware, but it does work. I'm running a RX7600 - 8GB myself, and happily doing SDXL. The main problem is that exhausting VRAM causes instability. Exceed it by a lot, and everything is handled fine, but if it's marginal... problems ensue. The AMD engineers are actively making the experience better, and it may not be long before it's a practical alternative. If/When that happens nVidia will need to slash their prices to sell anything in this sphere, which I can't really see themselves doing.
  - zargon 3y ago
    > If/When that happens nVidia will need to slash their prices to sell anything in this sphere It's just as likely that AMD will raise prices to compensate.
    - weebull 3y ago
      You think they're going to say "Hey, compute became competitive but nothing else changed performance therefore... PRICE HIKE!"? They don't have the reputation to burn in this domain for that IMHO. Granted you could see a supply/demand related increase from retailers if demand spiked, but that's the retailers capitalising.