7 ms·
Stability AI Launches Stable Diffusion XL 0.9
- soartificial 3y agoNGL I can't wait to get hold of this model file and run it locally, ill be sure to do a write up on it on my AI blog https://soartificial.com https://soartificial.com. I just hope that my GPU can handle this locally. I don't think 8gb of vram is going to be enough, might have to tinker with some settings. Im just looking forward to the custom LoRA files we can use with it :D
- GaggiX 3y agoI'm already seeing the speedrun to implement this model architecture in the automatic1111 webui.
- swyx 3y agoimplementing model architecture and supporting it in a web ui are two different things, maybe a link to pr?
- GaggiX 3y agoDo you mean that supporting the model in automatic webui is much more difficult than just implementing the model in a repo? I guess.
- brucethemoose2 3y agoThe A1111 backend is kinda not set up for this, as it is built around the old Stability AI 1.5/2.1 implementation (not HF diffusers which most other backends use). It would basically be a rewrite, if I were to guess... And at that point they mind as well port everything to diffusers.
- ilaksh 3y agoSo it's non-commercial but they are adding it to the API on Monday? Does that mean it will be commercial then?
- minimaxir 3y agoClipdrop is owned by Stability AI and you can access the model now, with its own API: https://clipdrop.co/stable-diffusion https://clipdrop.co/stable-diffusion The Stability AI API/DreamStudio API is slightly different. Yes, it's confusing.
- candiodari 3y agoThis is copyright: non-commercial for everyone that copies the model from them. Not for the ones producing their own model.
- binarymax 3y ago“ The model can be accessed via ClipDrop today with API coming shortly. Research weights are now available with an open release coming mid-July as we move to 1.0.” I read this as: commercial use through our API now, self hosted commercial use in July.
- EGreg 3y agoIs there image + text to image prompting? Can I do dreambooth here? If so, what commands do I use?
- brucethemoose2 3y agoYou can do this in SD 1.5/2.1 already. Encode the image like you do for pix2pix, and the text, then average those latents together. Dreambooth is gonna require an A100 I think... I doubt it will work on the free (16GB VRAM) Colab instances.
- bbor 3y agoFor anyone dumb like me: this is NOT a routine announcement, definitely read it through. The improvements they show off are honestly stunning. Also, they’ve (re-) established a universal law of AI: fuck it, just ensemble it
- isoprophlex 3y agoApparently that's what GPT4 is too: eight GPT3.5's ensembled together. Not sure if true, sounds plausible tho.
- EGreg 3y agoHow does ensembling work? What
- minimaxir 3y agoCombining the results of multiple models and then adding another layer onto the combined output tends to increase accuracy / reduce error rates. (not new to AI: it's been done for over a decade)
- liuliu 3y agoMany ways, for one: https://magicfusion.github.io/ https://magicfusion.github.io/
- vm90 3y agoCurious to know how this compares to mid journey v5
- vitorgrs 3y agoMidjourney IMO still better (especially because it can do hands), but this actually come pretty close. I created some amazing pictures with it already!
- nickthegreek 3y agoUntil it gets a control net release, MJ will be better.
- orbital-decay 3y agoIt's not better out of the box for text to image, this was known for quite some time. However, as soon as they release the weights (in a month as they promise), it will benefit from the tooling available for SD, without being limited to text to image. It's also a foundational model, not a finished product, and MJ will possibly use it, like they did in v4 with SD 1.5.
- Klaus4 3y agoMJv4 is not related to SD at all
- andybak 3y agoGot a link for that? I'm genuinely asking as I didn't know this.
- orbital-decay 3y agoI might be misremembering, but didn't they announce in their twitter that they were using SD somehow for MJ v4? Later deleting this with a bunch of other tweets.
- Klaus4 3y ago
- jdalgetty 3y agoIs the mac studio with the Apple M2 Max with 12‑core CPU, 30‑core GPU enough for something like this?
- minimaxir 3y agoLikely not. Even with earlier models performance may be finnicky: https://huggingface.co/docs/diffusers/optimization/mps https://huggingface.co/docs/diffusers/optimization/mps Additionally, for Apple Silicon you likely need 64 GB RAM (since CPU/GPU memory is shared) which is expensive.
- pkage 3y agoI’ve had good results with SD1.4/2 with MPS acceleration on similar hardware (M1 Max, though with 64gb). No stability issues with MPS, either. I’d say don’t rule it out just yet.
- SkyPuncher 3y agoI have an M2 MBP with 64 GB RAM. Performance with the older models is very good in my opinion. It feels to run faster locally than DreamStudio. I don't have benchmarks, but in any case the performance is not bad.
- itake 3y agoMy guess is you will need cuda support until someone ports it to mips
- jrflowers 3y agoSystem requirements Despite its powerful output and advanced model architecture, SDXL 0.9 is able to be run on a modern consumer GPU, needing only a Windows 10 or 11, or Linux operating system, with 16GB RAM, an Nvidia GeForce RTX 20 graphics card (equivalent or higher standard) equipped with a minimum of 8GB of VRAM. Linux users are also able to use a compatible AMD card with 16GB VRAM. I’m guessing that it will work eventually, though I’m not sure who will make that happen.
- brucethemoose2 3y ago
- bogwog 3y ago> Nvidia GeForce RTX 20 graphics card (equivalent or higher standard) RIP my 1080 TI. Does anyone know what specific feature they need which 20+ cards have and older ones don't?
- binarymax 3y agoNot enough RAM in your 1080TI Edit - it’s not the RAM. 1080TI has 11GB and this press release says it requires 8. So I’m going to speculate that it’s because 1080 lacks tensor cores compared to the 20x’s Turing architecture
- nolok 3y agoFunny knowing the stupidity nvidia is pulling with the 4xxx series regarding ram amount.
- deleted 3y ago[deleted]
- cma 3y agoSince it is now split into two models to do the generation, you could load one and do the first stage of a bunch of images, then load the second and complete them, with half the vram usage.
- ShamelessC 3y agoI believe the HF pipeline can do this already and I assume each stage uses more than 4 GB vram. There are other tricks they open source community will come up with though.
- brucethemoose2 3y ago8 bit (and 4 bit?) quantization is low hanging fruit, assuming its not already running in 8 bit. An 8GB requirement kinda sounds like they have already quantized the model though.
- xigency 3y agoThe hands look better but there’s still hints of a sixth finger in each of them.
- krunck 3y agoThe last image has big toes for thumbs.
- philshem 3y agohttps://en.wikipedia.org/wiki/Brachydactyly_type_D https://en.wikipedia.org/wiki/Brachydactyly_type_D
- seydor 3y agoand 5 phalanges
- brucethemoose2 3y agoNothing a hands LORA can't fix.
- xigency 3y agoTrue, but you still won’t get a coherent picture of someone picking their nose I bet.
- heliophobicdude 3y agoIf Emad tweeted this image [1] made with SDXL, then text in image could possibly better! 1:https://twitter.com/emostaque/status/1671885525639380992 https://twitter.com/emostaque/status/1671885525639380992
- jrflowers 3y agoThat’s probably DeepFloyd.
- gwern 3y agoText will be better due to simple scale, but the text will still be limited due to the use of a CLIP for text encoding (BPEs+contrastive). So that may be SD XL 0.9 but it should still be worse due to not using T5 like https://github.com/deep-floyd/IF https://github.com/deep-floyd/IF
- varelse 3y ago[dead]
- joshuahedlund 3y agoAm I the only one who doesn't see an obvious difference in the quality between the left and right photos? (Maybe the wolf one) And these are extremely-curated examples!
- HelloMcFly 3y agoThe wolf looks better, but also looks less like what you'd see in a "nature documentary" (part of the prompt). I think the coffee cup looks better in the right phot, it seems a tad bit more real to me. Like you I much prefer the alien photo on the left, but the photos are so stylistically different I'm not sure that says anything about the releases' respective capabilities.
- itairall 3y agoNo, I have generated a few thousand midjourney images and there is quite a difference in these images actually. It is hard to describe but there is a very unnatural "sheen" to the images on the left. The SDXL 0.9 images look more photo realistic but they still aren't quite at the level that midjourney can do. The best example is the wolf's hair between the ears in the SDXL 0.9 image. It is just a little too noisy and wavey compared to how a real wolf photo would look. Midjourney 5.1 --style raw would still handily beat this image if making a photo realistic wolf. The jacket on the Alien in the SDXL 0.9 image also has too much of that AI sheen but it kind of works in this image as an effect for the jacket material so not really the best example. The coffee cup isn't very good on either of them IMO. The trees on the right are still not blurred quite right. They are hiding the hand with this image on the right too. You can see how bad the little and ring finger is on the left image. Obviously, this is all very nit picky.
- vitorgrs 3y agoThat's because they are actually comparing old version of SDXL vs new one. The old version already improved things... The real comparison should be with SD 1.5/2.1, and is WAY better.
- poulpy123 3y agoAt the resolution provided they are indeed very close. In my eyes: In the first example, the second image is more representative of Las Vegas for the foreigner I am, but none of them hav ethe scratchy found film requirement In the second example, both fit the prompt, but the first image look more coming from a documentary than the second one in the third example, the hand from the second picture looks much better
- LorenDB 3y agoAny speculation why the AMD cards require twice the VRAM that Nvidia cards do? I have an RX 6700 XT and I'm disappointed that my 12 GB won't be enough.
- brucethemoose2 3y agoYou will want an optimized implementation from torch-mlir or apache tvm anyway. We had this for SD 1.5, but it always stayed obscure and unpopular for some reason... I hope its different this time around.
- jacooper 3y agoYour 6700XT won't work anyway, since its not supported by ROCm
- brucethemoose2 3y agoThe 6000 series "unofficially" works with rocm, and that is hopefully getting more official.
- dharmab 3y agoIt works on my 6900XT with ROCm
- jacooper 3y agoBecause anything higher the 6800 is unofficially supported
- kreig 3y agoThat's not accurate, I was able to make my old RX 5500XT work with ROCm some months ago, only after 6 hours of compilation process though...
- jacooper 3y agoIts not supported, only navi 21 is supported, which is RX 6800+ https://github.com/RadeonOpenCompute/ROCm/issues/1668 https://github.com/RadeonOpenCompute/ROCm/issues/1668
- xnx 3y agoThe progress in AI is great, but very hard to keep up with (or understand). It's real to me once it makes it into Automatic1111.
- brucethemoose2 3y agoSmall rant: I don't like how SD consolidated around the A1111 repo. The features are great, and it was fantastic when SD was brand new... but the performance and compatibility is awful, the setup is tricky, and it sucked all the oxygen out of the room that other SD UIs needed to flourish.
- WithinReason 3y agoHighly recommended: https://github.com/comfyanonymous/ComfyUI https://github.com/comfyanonymous/ComfyUI
- ShamelessC 3y agoIt’s open source. The only way to compete is on your merits in open source. If you want another UI to flourish, clone both it and A111, copy and paste the bits from A111 you’d like to have in yours (with attribution) and push it up along with any features you personally want. That does require developer time, and developers may converge on a popular implementation with good tests and lots of features as it’s easier to contribute. The bottleneck isn’t really the community though, it’s the developers.
- brucethemoose2 3y agoIts not that simple, as A1111 uses the old stability AI implementation while pretty much everything else uses HF diffusers code. I worked trying to add torch.compile support to A1111 for a bit, fixing some graph breaks locally, but... It was too much. Some other things, like ML compilation backends, are also basically impossible.
- Filligree 3y ago
- brucethemoose2 3y agoWhat's the dataset? How commercially viable/legally questionable is it? This is critical, for legality of use, ethics concerns, and the quality of the output (as overly zealous filtering can degrade the model like it did for SD 2.0).
- xigency 3y agoIf I recall correctly, Stability AI’s process to skirt copyright is to have the training data compiled and model weights trained by a third-party university. Educational research institutions have more lax requirements around copyright. That may or may not be a legitimate way to work under existing laws, but doesn’t tell us much about what the moral/ethical/legal considerations should be, which seems like an open question.
- pmayrgundter 3y agoThat sounds more like their story for SD 1.5 last year. I think there was some kerfuffle between Stability.ai and Runway.ai/Heidelberg Uni (see the Forbes article; won't link as I'm unclear on the veracity), who they were working with, and they may have parted ways by their first indie work on SD 2.x around the holidays. Either way, the Uni connection story may be old.
- homarp 3y agohttps://news.ycombinator.com/item?id=36185891 https://news.ycombinator.com/item?id=36185891 discusses the Forbes article
- homarp 3y agoand I guess you can add https://www.bloomberg.com/news/articles/2023-06-26/stability-ai-head-of-research-resigns-from-startup https://www.bloomberg.com/news/articles/2023-06-26/stability... now
- djbusby 3y agoSo, there are at least a few dozen AI image generating sites, some specialized, other not. Are they all powered by SD? Maybe just with some better pre-prompting? Or are there other engines (eg DALL-E)?. AFAIK only SD can be run locally?
- vitorgrs 3y agoMost sites/apps use Stable Diffusion, yes. Custom model of these, because the base model of SD 1.5/2.1 is really... heh, complicated to say the least.
- pmayrgundter 3y agoI've only run across 3 primary models: Midjourney (via their Discord), Dall-E and SD. And yes, there's a bunch of sites, but I've seen very similar quality to SD and no mention yet of a different base. I do expect there are other bases out there, but haven't seen any of quality yet. Before this release (XL 0.9) it's been unclear how much of the SD quality was in-house or came from their prior collab with Runway/Heidelberg.
- andybak 3y agoKandinsky, Deep Floyd... Also Midjourney is derived from SD I believe.
- pmayrgundter 3y agoI don't think MJ is from SD.. found no mention of SD on their site or on Wiki, besides a comparison. Any citation?
- famouswaffles 3y agoNone of those is derived from SD
- andybak 3y agoI wasn't saying they were. Read back one more message. (However, I thought Midjourney definitely was at some point)