12 ms·
Stable Diffusion XL 1.0
- badwolf 3y agoCan SD draw hands finally?
- GaggiX 3y agoYou can already easily generate images with good looking hands if you use a good custom model.
- vitorgrs 3y agoWith a few tries, yes. Probably will be even better with negative embedding.
- dragonwriter 3y ago> Probably will be even better with negative embedding. And/or hand-specific LoRa and/or a workflow using something like ADetailer extension in A1111 that applies a model to recognize hands [0] and then inpaints them. [0] recognition models are also provided for people, faces, and eyes, too, and it can use additional custom models for other things.
- MasterScrat 3y agoIt’ll be "released" once the model weights show up on the repo or in HuggingFace… for now it’s "announced" It should appear here at some point, currently only the VAE was added: https://huggingface.co/stabilityai https://huggingface.co/stabilityai
- nickthegreek 3y agoIt does appear to be live on Clipdrop. https://clipdrop.co/stable-diffusion https://clipdrop.co/stable-diffusion
- deleted 3y ago[deleted]
- ftufek 3y agoThe release event is in like ~30 minutes on their discord, probably the announcement went out a bit early.
- naillo 3y agoYou get access to the weights instantly if you apply for them. It's basically not a hurdle. (I've been having fun with this for a few days. https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9 https://huggingface.co/stabilityai/stable-diffusion-xl-base-... Not sure there's much of a difference with the 1.0 version.)
- MasterScrat 3y agoFor 1.0? Where do you apply? Or are you talking about 0.9?
- Ukv 3y agoThe ones you can apply for access to are the 0.9 weights, which have been available for a couple of weeks. Unless the SDXL 1.0 weights are also available by application somewhere that I'm unaware.
- taminka 3y agohttps://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0 https://huggingface.co/stabilityai/stable-diffusion-xl-base-... :)
- quartz 3y agoIsn't this it? https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0 https://huggingface.co/stabilityai/stable-diffusion-xl-base-...
- MasterScrat 3y agoYes, it's now been released
- fernly 3y agoIt only replies, "module 'diffusers' has no attribute 'StableDiffusionXLPipeline'"
- deleted 3y ago[deleted]
- andybak 3y agoIn the meantime I've been getting good mileage out of Kandinsky - anyone got a good sense of how they compare?
- brucethemoose2 3y agoThis is the first I have heard of Kandinsky. Thanks for the tip. SDXL is a bigger model. There are some subjective comparison posts with SDXL 0.9, but I can't see them since they are on X :/
- simbolit 3y agoThat sounds so weird, it took me a minute to understand. Go to nitter.net which has no login requirement and no ads, but all the same content that X (tmsfkat) has.
- brucethemoose2 3y agoTwo Nitter instances failed to load it, unfortunately. And yeah, X is weird to type out too.
- jrflowers 3y agoI hope someday there’s a version of this or something comparable to it that can run on <8gb consumer hardware. The main selling point of Stable Diffusion was its ability to run in that environment.
- minsc_and_boo 3y agoI feel like this is the greatest demand for LLMs at the moment too. It's hard to believe we're only 8 months into this industry, so I imagine we'll start seeing smaller footprints soon.
- simbolit 3y ago8 months from what point? Gpt3 is 36 months old. Dalle-e is 28 months old. Even StableDiffusion is like 11 months old.
- minsc_and_boo 3y agoFair, I should have said 8 months since the market exploded.
- brucethemoose2 3y agoWe already do. MLC-LLM and Llama.cpp have Vulkan/OpenCL/Metal 3 bit implementations. That can run llama 7B (or maybe even 13b?) in 8GB. TBH devices just need more ram for coherent output though. Llama 13b and 33b are so much "smarter" and more coherent than 7B with 3 bit quant.
- TeddyDD 3y ago13b q5 llama report „total VRAM used: 8321 MB” so 3bit will most likely fit into 8GB.
- naillo 3y agoYou can do this if you select the `pipe.enable_model_cpu_offload()` option. See this https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9 https://huggingface.co/stabilityai/stable-diffusion-xl-base-...
- weird-eye-issue 3y agoNot actually released in the API unlike they said.
- weird-eye-issue 3y agoIt took several hours after it was announced that it was already available in the API
- tmaly 3y agoI will wait for the automatic1111 web ui version
- Der_Einzige 3y agoIt's already supported in automatic1111 (see recent updates), and someone in the community will convert it to the automatic1111 format within minutes/hours after it's released on huggingface.
- deleted 3y ago[deleted]
- seydor 3y agowhats the memory usage of sdxl ?
- SV_BubbleTime 3y agoIt depends greatly on your UI and the size you are generating. There is no hard answer.
- k12sosse 3y agoRuns great on my 10GB 3080 FTW3. ComfyUI moreso than auto1111.
- l33tman 3y agoAuto1111 (latest on git) OOMed my 3080 running the base-xl on 1024x1024 unfortunately. Granted, my xorg takes up almost 950 MB on the precious VRAM. Did you get it to run using A1111 on the FTW3 without OOMing it?
- andrewmunsell 3y agoBeen working fine on a 8 GB 3070 generating 1024x1024 images, using Comfy UI with the refiner
- 3y ago
- vouaobrasil 3y agoThis explosion of AI-generated imagery will result in an explosion of millions of fake images, obivously. Perhaps in the short-term this is fun, but in the long-term, we will lose a bit more scarcity, which is not that great in my opinion. Isn't the best part of a meal eating after you've not had anything to eat for a while? The best part about a kiss that you've quenched the pain of missing your partner? The best part of art is that you haven't seen anything good in a while? Scarcity is an underappreciated gift to us, and the relative scarcity per capita is in a sense what drives us to connect with other people, so that we may be priveleged to witness the occasional spark of creativity from a person, which in turn tells us about that person. Although that sort of viewpoint has been declining for some time due to the intensely capitalistic squeezing of every sort of human endeavor, AI brings this to a whole new level. I think if those making this software thought a bit about this, they might second-guess whether it is truly right to release it. Just a thought.
- RcouF1uZ4gsC 3y agoI want to appreciate your comment, but I can't. Can you please chisel it on stone tablets for me? That will really help me appreciate it.
- soligern 3y agoEnforcing artificial scarcity is idiotic and counter progressive. There will be other things that will continue to be uncommon that humans will continue to appreciate. This is what human progress looks like. Imagine someone said this when agriculture started up- “The great thing about fruits and vegetables is that they taste so sweet the few times we find them. We shouldn’t grow them in bulk”
- __loam 3y agoAI Art models are completely dependent on human labor to function. "Out-competing" human generated images will damage the commons and make it harder to train these models over time as they push human creative labor out of the market, if we believe it's even competitive. Personally I think this guy has a point and he's pointing to something that I don't believe a lot of ai art advocates have considered: the attention economy. There isn't actually any scarcity in the current market for artistic content. There are literally millions of people producing art every day, and the market is very winner take all. There are few artists who are able to support themselves on their work, and their skills are exceptional and specifically in demand. My theory is that the supply of AI generated content will be so vast, and the perception around it will be that it's low effort and low quality, that so called AI artists are going to have trouble distinguishing themselves in a market where they're saturating it and all using the same models. I think your perception of this market is flawed. Art is not a fungible good like food or clothing.
- naillo 3y agoStability AI is awesome I love them
- freediver 3y agoI am completely uninformed in this space. Would someone be kind to explain what the current state of the art in image generation is (how does this compare to Midjourney and others)? How do open source models stack up? Also what are the most common use cases for image generation?
- liuliu 3y agoSDXL 0.9 should be the state-of-the-art image generation model (in the open). It generates at 1024x1024 large resolution, with high coherency and good selection of styles out of box. It also has reasonable text-understanding comparing to other models. That has been said, based on the configurations of these models, we are far from saturating what the best model can do. The problem is, FID is terrible metrics to evaluating these models so like LLM, we are a bit clueless about how to evaluate them now.
- GaggiX 3y agoWhy do you think FID is a terrible metrics? What don't you like in particular about it?
- liuliu 3y agoI overspoke. FID is a fine metrics to observe the training progress of your own model. And it correlates well with some coherency issues of generative models. But for cross model comparisons, especially for models that generally do well under FID, it is not discriminative enough to separate better / good.
- deleted 3y ago[deleted]
- orbital-decay 3y agoSDXL is in roughly the same ballpark as MJ 5 quality-wise, but the main value is in the array of tooling immediately available for it, and the license. You can fine-tune it on your own pictures, use higher order input (not just text), and daisy-chain various non-imagegen models and algorithms (object/feature segmentation, depth detection, processing, subject control etc) to produce complex images, either procedural or one-off. It's all experimental and very improvised, but is starting to look like a very technical CGI field separate from the classic 3D CGI.
- ShamelessC 3y agoI thought this release had been announced already? Or was that not 1.0? Could have sworn they released an "XL" variant a little while ago?
- GaggiX 3y agoIt was the research weights of the v0.9 model
- deleted 3y ago[deleted]
- PeterStuer 3y agoLet's see wether derived models will suffer less from the 'same face actor'-model response to every portrait prompt. It's not trivial to get photoreal models not lookalike without resorting to specific, typically celeb based, finetunes.
- jamesdwilson 3y agoStill can't draw hands correctly it looks like.
- naillo 3y agoI can't say for 1.0 but in 0.9 hands get fairly often rendered perfectly. It's not always right but it's way better than any other earlier release (where it's usually consistently wrong).
- dragonwriter 3y agoBoth 0.9 and 1.0 seem to frequently merge fingers when hands are clasped; they don’t do nearly as much of other errors with them as earlier SD models.
- wincy 3y agoEh it’s not a huge deal with stable diffusion because you can inpaint. So you mask out the bad hands and generate a few dozen iterations that merge perfectly with the rest of the image. You’re bound to get something that looks good. With SD2.1 I’d generate 100 or so images using inpainting and the “good fingers” was about 1% hit rate. If it’s up to 10% that’d be great because generating 10 images takes just a few seconds on an A100
- hhjinks 3y agoHands are generally a non-issue at this point. You can just inpaint them and use a negative prompt LoRA to get good hands in just a few attempts. That is, if you don't just get good enough hands right away. That happens surprisingly often.
- lacker 3y agoI'm out of date on the image-generating side of AI, but I'd like to check things out. What's the best tool for image generation that's available on a website right now? Ie, not a model that I have to run locally.
- PUSH_AX 3y agoMidjourney right? Although, discord isn't a website I guess.
- iambateman 3y agoProbably Midjourney, but I like Dreamstudio better.
- the_lonely_road 3y agohttps://playgroundai.com/create https://playgroundai.com/create Not affiliated in anyway and not very involved in the space. I just wanted to generate some images a few weeks ago and was looking for somewhere I could do that for free. The link above lets you do that but I suggest you look up prompts because its a lot more involved than I expected.
- aaarrm 3y agoAny particularly useful resources for looking into prompts?
- brucethemoose2 3y agoThis AI Horde UI has, IMO, some really good templates and suggestions: https://tinybots.net/artbot https://tinybots.net/artbot
- davely 3y agoHey! Creator of ArtBot here. Thanks for plugging the site! For those not aware, here's an interesting fact about ArtBot (and the AI Horde in general) -- we've been running an A/B test with Stability.ai for the last 3 weeks or so related to SDXL [1]. Any time a user generates an image using SDXL_beta on the AI Horde, they get two images back. They pick which image they think is best for the given prompt. This data is sent back to Stability.ai in order to help improve their image models. In a similar vein, LAION partnered with the AI Horde earlier this year in order to gather aesthetics ratings for improving various image datasets. [2] It's a cool little open source community and there's just a ton of stuff going on. [1] https://dbzer0.com/blog/stable-diffusion-xl-beta-on-the-ai-horde/ https://dbzer0.com/blog/stable-diffusion-xl-beta-on-the-ai-h... [2] https://laion.ai/blog/laion-stable-horde/ https://laion.ai/blog/laion-stable-horde/
- accrual 3y agoIt sounds like after the previous 0.9 version there was some refining done: > The refining process has produced a model that generates more vibrant and accurate colors, with better contrast, lighting, and shadows than its predecessor. The imaging process is also streamlined to deliver quicker results, yielding full 1-megapixel (1024x1024) resolution images in seconds in multiple aspect ratios. Sounds pretty impressive, and the sample results at the bottom of the page are visually excellent.
- Tenoke 3y agoThey have bots in their discord for generating images bases on user prompts. Those randomize some settings, compare candidate models and are used for rlhf fine-tuning and that's the main source of refining which will continue even after release.
- dragonwriter 3y agoThere were, IIRC, three different post-0.9 candidate models in parallel testing to become 1.0 recently.
- latchkey 3y agoAmazing that their examples at the bottom of the page still show really messed up human hands.
- mynameisvlad 3y agoSome of them look surprisingly correct, so it looks like there's been at least some progress on that front. I would assume these are among the best examples of many, many attempts so it still seems to be a ways off.
- vitorgrs 3y agoI for sure generated hands better than they showed :)
- HelloMcFly 3y agoI've personally observed that the drawing of hands in Midjourney and SD has been getting incrementally better release after release.
- k12sosse 3y agoHands being bad is a result of people one shotting images, you need to go repaint them afterwards I've found. But it'll do it great if you inpaint well.
- RobotToaster 3y agoIs this pre-censored like their other later models?
- Remmy 3y agoYes.
- epups 3y agoNot sure why this is downvoted. This model is heavily censored by default.
- naillo 3y agoI've been playing with 0.9 and it can generate nude people so it seems not.
- Auracle 3y agoNo. From what I’ve gathered was trained on human anatomy, but not straight up porn. What they tried for 2.0/2.1 was way too overdone, to the point where if I prompted “princess Zelda,” the generation would only look mildly like her. Presumably they just didn’t have many images of people in the training. 1.5 and SDXL both work fine of that front. Fine tuners will quickly take it further, if that’s what you’re after.
- gettodachoppa 3y agoI don't think 1.x was trained on porn either? I seem to remember the issue with 2.x is that they removed all the commercial art from top-notch illustrators from the training data due to the backlash, so it was just way worse at generating great-looking things, which is all the user cares about. So the community stayed on their custom-trained models derived from SD 1.5 (which, yes, often included porn).
- Auracle 3y ago2.0 included a filter on the training data that removed all nudity. It went way too far and removed a lot of humans. They tried to rectify it a bit with 2.1 but even that was still hampered. What you said also happened, but the main thing was the base model didn't have a great concept of human anatomy. Apparently it was really hard to train for anything else as well.
- amilios 3y agoI always wondered why the vision models don't seem to be following the whole "scale up as much as possible" mantra that has defined the language models of the past few years (to the same extent). Even 3.5 billion parameters is absolutely nothing compared to the likes of GPT-3, 3.5, 4, or even the larger open-source language models (e.g. LLaMA-65B). Is it just an engineering challenge that no one has stepped up for yet? Is it a matter of finding enough training data for the scaling up to make sense?
- naillo 3y agoThey often reference this paper as the motivation for that https://arxiv.org/pdf/2203.15556.pdf https://arxiv.org/pdf/2203.15556.pdf I.e. training with 10x data and 10x longer can yield as good models as a gpt-3 model but with fewer weights (according to the paper) and the same principle applies in vision.
- brucethemoose2 3y agoDiffusion is relatively compute intensive compared to transformers llms, and (in current implementation) doesn't quantize as well. A 70B parameter model would be very slow and vram hungry, hence very expensive to run. Also, image generation is more reliant on tooling surrounding the models than pure text prompting. I dont think even a 300B model would get things quite right through text prompting alone.
- amilios 3y agoHmm this is a good point, diffusion requires several (many?) inference passes as you refine the noise into an image, right? Makes sense that this is more expensive to scale up. Thanks for the explanation!
- brucethemoose2 3y agoTechnically the llms require a pass for each token, but the passes are cheaper and benefit more from batching.
- 3y ago
- mkaic 3y agoThe official blog post from Stability is finally up and would probably be a better URL to link to than the TechCrunch coverage: https://stability.ai/blog/stable-diffusion-sdxl-1-announcement https://stability.ai/blog/stable-diffusion-sdxl-1-announceme...
- thepaulthomson 3y agoMidjourney is still going to be hard to beat imo. Comparing SD to MJ is a little unfair considering their applications and flexibility, but I do really enjoy the "out of the box" experience that comes with MJ.
- hospitalJail 3y agoMJ quality is significantly worse. Everything has the Pixar look and barely follows the prompt. Its nice for a toy, but SD with Automatic1111 is miles ahead of MJ.
- jyap 3y agoDifferent use case. I can run SDXL 1.0 offline from my home. I can’t do this with Midjourney. A closed source model that doesn’t have the limitation of running on consumer level GPUs will have certain advantages.
- starik36 3y agoWhat type of setup do you have at home? What type of GPU? MJ completes a pretty high quality photo in about a minute. Does SD compare?
- BrentOzar 3y agoWith an RTX 4090, you can crank out several images per minute, even at high resolutions.
- brnaftr361 3y agoI haven't done it in a while but I was cranking images out at 11s/output on a 3080. But it depends on your workflow, too. I started low res/low samples (32-64) and scaled up or used recursion until I got a desirable result or found a nice seed. I think I was doing 512x916 or something close to that.
- elishah 3y agoWith SD you have a lot of control over not just basics like image size and prompt complexity, but also things like how many iterations of which different sampler(s) get used. So speed can vary wildly depending on how you're choosing to use it. And that's without even getting into the wide variance of hardware. But generally speaking, it will usually be significantly faster than one image per minute.
- mt3ck 3y agoIs there anything like this for the vector landscape? This may just be due to the iterative denoising approach a lot of these models take but they only seem to work well when creating raster style images. In my experience when you ask them to create logos, shirt designs, illustrations, they tend to not work as well and introduce a lot of artifacts, distortions, incorrect spellings etc.
- cheald 3y agoA lot of people are having success by adding extra networks (lora is the most common) which are trained on the type of image you're looking for. It's still a raster image, of course, but you can produce images which look very much like rasterizations of vector images, which you can then translate back into SVGs in Inkscape or similar.
- orbital-decay 3y agoIf you mean raster images that look like vector and contain arbitrary text and shapes, controlnets/T2I adapters do work for this. You could train your custom controlnet for this, too. (it requires understanding) As for directly generating vector images, there's nothing yet. Your best bet is generating vector-looking raster and tracing it.
- ilaksh 3y agoThere are SD models tunes for vector like raster output. And XL has specifically focused on this use case as one of the improvements. Try SDXL 1 on Clipdrop or Dreamstudio.
- lee101 3y ago[dead]
- skybrian 3y agoI tried it in dreamstudio. Like all the other image generators I've tried, it's rubbish at drawing a piano keyboard or an accordion. (Those are my tests to see if it understands the geometry of machines.) A couple of accordion pictures do look passable at a distance. Another test: how well does it do at drawing a woman waving a flag? One thing that strikes me is that it generates four images at a time, but there is little variety. It's a similar looking woman wearing a similar color and style of clothing, a similar street, and a large American flag. (In one case drawn wrong.) I guess if you want variety you have to specify it yourself? AI models seem to be getting ever better in resolution and at portraits.
- methyl 3y agoMy go-to test is "elephant riding unicycle". Neither Midjourney nor Stable Diffusion XL is capable of doing this.
- blagie 3y agoIt's often said porn drives technology. I clicked through the links in the article, since they sounded technically interesting. They led to AI-generated porn. Those, in turn, led to pages about training SD to generate porn. Now, two disclaimers: 1) I am not interested in AI-generating porn 2) I haven't followed SD in maybe 6-9 months With those out-of-the-way, the out-of-the-box tools for fine-tuning SD are impressive, well beyond anything I've seen in the non-porn space, and the progress seems to be entirely driven by the anime porn community: https://aituts.com/stable-diffusion-lora https://aituts.com/stable-diffusion-lora 10 images is enough to fine-tune. 30-150 is preferred. This takes 15-240 minutes, depending on GPU. I do occasionally use SD for work. If this works for images other than naked and cartoon women, and for normal business graphics, this may dramatically increase the utility of SD in my workflows (at least if I get around to setting it up). I want my images to have a consistent style. If I'm making icons, I'd like to fine-tune on my baseline icon set. If I'm making slides for a deck, I'd like those to have a consistent color scheme and visual language. Now I can. Thanks creepy porn dudes! The other piece: Anyone trying to keep the cat in the bag? It's too late.
- dragonwriter 3y ago> the progress seems to be entirely driven by the anime porn community: Its not entirely driven by porn communities, and the porn communities driving it aren’t entirely anime porn communities (and the anime communities driving it aren’t entirely porn communities.) But, yeah, the anime + porn/fetish art + furry + rpg art + scifi/fantasy art communities, and particularly the niches in the overlap of two or more of those are, pretty significant. > If this works for images other than naked and cartoon women It does, and while it may not be large proportionally compared to the anime-porn stuff, there’s a lot of publicly distributed fine tuned checkpoints, LoRas, etc., demonstrating that it does.
- Auracle 3y agoIt absolutely works for things other than naked and cartoon women. Here are some generations of my daughter and dog (together!). I believe most of these are from a fine tuned model of them and not an extracted LoRA, though I use that sometimes too: https://imgur.com/a/naHgnel https://imgur.com/a/naHgnel
- trollo778899 3y ago[flagged]