61 ms·

Stable Diffusion 2.0

1368 points by hardmaru 4y ago

nessus42 4y ago
There's nothing in the release notes that says whether 2.0 can do hands without a 99% chance of producing deformed results.
minimaxir 4y ago
GitHub Repo: https://github.com/Stability-AI/stablediffusion https://github.com/Stability-AI/stablediffusion HuggingFace Space (currently overloaded unsurprisingly): https://huggingface.co/spaces/stabilityai/stable-diffusion https://huggingface.co/spaces/stabilityai/stable-diffusion Doing a 2.0 release on a (US) 2-day holiday weekend is an interesting move. It seems a tad more difficult to set up the model than the previous version.
- gst 4y ago
  > Doing a 2.0 release on a (US) 2-day holiday weekend is an interesting move. Their HQ seems to be located in London.
  - minimaxir 4y ago
    True, but it's going to dampen the launch a bit.
    - jehb 4y ago
      Is it? For me, I'm in tech but nowhere near anything for which playing with a new SD release would be relevant to my day job. Having a couple of extra days off to play with a new tech toy probably means I'll use it more.
    - smcleod 4y ago
      I don't see why? The world is a whole lot bigger than the USA!
- Operyl 4y ago
  Seems like a potentially good time to launch it, lots of young people with free time.
  - Gigachad 4y ago
    Less likely journalist whingers will pick it up too.
  - kennyloginz 4y ago
    Definitely something to talk about or share with the fam.
- Existenceblinks 4y ago
  I wonder why these AI repos' documentation are so bad compared to what we are used to in general. Where is intro/get started/example(commands)/config(docs) etc.
  - blihp 4y ago
    I believe it's a combination of the fact that most of these models are basically 'research dumps' primarily targeting other researchers and given this they are assuming a level of familiarity with related tools/libraries. So it's up to interested people in the community to take it the last mile/block/whatever to make it easy to use, address specific use cases etc. for use by a less academic/technical audience.
    - Existenceblinks 4y ago
      Yeah, generally speaking, it's not optimized for open collaboration (not even close .. pun intended).
- IanCal 4y ago
  I think it's looking fairly similar, the first one was a bit tricky too. Later improvements by the community made it clearer. The docs aren't good though, it tells you to download two things when actually I think you only need one. If you do need two then it doesn't tell you at all where to put the second. You really need xformers if you're doing it at home, I've got a 3090 and it blew through the ram without it. However, the instructions didn't work for me for compiling and there's an incompatibility if you try and install from conda. You can have it work but you need to upgrade python from 3.8.5 to 3.9 in the yaml file first, then you can install it (xformers needs 3.9+, and something else in SD breaks on 3.10+ so 3.9 works). This needs the classic "sit next to a new person installing it by following the docs and see what problems they hit, fix the docs and start from scratch again" process. Looks good, though so far the images I've made don't look as nice as with 1.4, but I guess that's largely down to finding the right tweaks for the model and right magic wording for the prompts.
rogers18445 4y ago
They apparently tried to combat NSFW generation by filtering the training dataset not to include any.
- throwup 4y ago
  You can generate all the bloody violent gore you like, but god forbid anybody see a human body in its natural state
  - echelon 4y ago
    There is worry about generating illegal content. If the model understands multiple concepts, it can combine them.
  - dang 4y ago
    "Avoid generic tangents." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- argella 4y ago
  Bummer. AI porn is fun.
  - deleted 4y ago
    [deleted]
  - Gigachad 4y ago
    The future is probably models trained almost exclusively on porn.
    - joshspankit 4y ago
      Porn has driven many tech advances. I predict that models trained on specific porn genres will appear as soon as training a good model is doable for under $5000. They’ll get here much quicker if we get video to that mark first.
satvikpendem 4y ago
Looks good. I've gotten bored with AI image generation these days however, after using a lot of SD the past few months. I suppose that's the hedonic treadmill in action.
- quickthrower2 4y ago
  Another thing to take for granted eh!
natch 4y ago
> greatly improves the quality of the generated images compared to earlier V1 releases That’s great!
- minimaxir 4y ago
  The repo provides a chart with a quantitative measure (FID/CLIP score) where the new 2.0 models do indeed have much better results than the earlier 1.5 model.
  - natch 4y ago
    Nice, will be interesting to see.
theylovezmw 4y ago
Interesting this is being marketed as a 2.0 release so quickly after the first version was launched. These new updates are quite great but are they so game-changing that it is considered 2.0?
- vanshg 4y ago
  It doesn't need to be game-chaning to be considered V2. It just needs to be "the next version"
LASR 4y ago
I am a solo dev working on a creative content creation app to leverage the latest developments in AI. Demoing even the v1 of stable diffusion to the non-technical general users blows them away completely. Now that v2 is here, it’s clear we’re not able to keep pace in developing products to take advantage of it. The general public still is blown away by autosuggest in mobile OS keyboards. Very few really know how far AI tech has evolved. Huge market opportunity for folks wanting to ride the wave here. This is exciting for me personally, since I can keep plugging in newer and better versions of these models into my app and it becomes better. Even some of the tech folks I demo my app to, are simply amazed how I can manage to do this solo.
- jnrk 4y ago
  > The general public still is blown away by autosuggest in mobile OS keyboards. And it's still not available in my language on iOS... :( (Norwegian)
- ksec 4y ago
  Last time we had people in the field of AI writing on HN expecting another AI Winter. I thought all the improvement we have now, and we might get in the future like Stable Diffusion 3.0 and possibly many others, while not L5 Self Driving Car or General AI, has yet to be fully used outside of Tech community, or even in Tech itself. The long tail of distribution and positive feedback loop will sustain its development for another 10 years.
- searchableguy 4y ago
  What are you building?
  - KidComputer 4y ago
    - wfme 4y ago
      This comment pretty clearly breaks the commenting guidelines https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html >Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Please don't fulminate. Please don't sneer, including at the rest of the community. Edit out swipes. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
knicholes 4y ago
Does anyone have a good source for all sorts of prompts for image generation?
- gedy 4y ago
  https://lexica.art https://lexica.art
- mromanuk 4y ago
  Have you seen lexica.art? There are a few others, but I don’t remember the URLs
- asicsp 4y ago
  https://publicprompts.art/ https://publicprompts.art/ is amazing
- jasfi 4y ago
  InventAI: https://inventai.xyz https://inventai.xyz is being developed, will help a lot with prompt engineering.
WatchDog 4y ago
Are the GPU memory requirements different for this release? Is now it possible to generate higher resolution images with less memory?
- mugivarra69 4y ago
  dont thinkso, ran on my macbook pro, same footprint. idk cuda side.
  - swyx 4y ago
    how do you check footprint?
wyldfire 4y ago
depth2img looks really interesting. I was thinking that someone should train an art model like SD on 3d models+textures. This isn't quite that but it seems like it gets some of that effect.
wyldfire 4y ago
This one's publicly downloadable? I think I must've missed 1.5. it had been postponed for a while (for good reasons discussed throughout threads here) and I didn't notice whether it had been released.
- minimaxir 4y ago
  1.5 was released, although it was not dramatically better than 1.4 (outside of better inpainting) so it didn't get much buzz. https://huggingface.co/runwayml/stable-diffusion-v1-5 https://huggingface.co/runwayml/stable-diffusion-v1-5
liuliu 4y ago
Seems the structure of UNet hasn't changed other than the text encoder input (768 to 1024). The biggest change is on the text encoder, switched from ViT-L14 to ViT-H14 and fine-tuned based on https://arxiv.org/pdf/2109.01903.pdf https://arxiv.org/pdf/2109.01903.pdf. Seems the 768-v model, if used properly, can substantially speed-up the generation, but not exactly sure yet. Seems straightforward to switch to 512-base model for my app next week.
- millimeterman 4y ago
  I'm disappointed they didn't push parameter count higher, but I suppose they want to maintain the ability to run on older/lower end consumer GPUs. Unfortunately it severely limits how high-quality the output can be.
  - ShamelessC 4y ago
    Can’t forget time it takes to run inference, even on the latest A100/H100. Generating in under e.g. ten seconds enables more use cases (and so on until high fps video is possible).
  - naillo 4y ago
    They're motivating that choice via this paper: https://arxiv.org/pdf/2203.15556.pdf https://arxiv.org/pdf/2203.15556.pdf The paper shows that you can get better performance than gpt-3 with a much smaller model if you bump up the training time and training data like x4.
    - macrolime 4y ago
      Larger models are still much better. Google's parti model can do text perfectly and follows prompts way more accurately than Stable Diffusion. It's 20B parameters and with the latest int8 optimizations it should be possible to get that running on a consumer 24GB card in theory. I think they're looking into larger models later though
- smeagull 4y ago
  Oh shit, I think that means it's CPU only for me now.
dzink 4y ago
This Thanksgiving, I would like to extend a very warm Thank You to the Stable Diffusion team! Side note: The 4x upscaler model is showing as unavailable if you follow the hugging face link to it.
nl 4y ago
Highlights: 768x768 native models (v1.x maxed out at 512x512) a built-in 4x upscaler: "Combined with our text-to-image models, Stable Diffusion 2.0 can now generate images with resolutions of 2048x2048–or even higher." Depth-to-Image Diffusion Model: "infers the depth of an input image, and then generates new images using both the text and depth information." Depth-to-Image can offer all sorts of new creative applications, delivering transformations that look radically different from the original but which still preserve the coherence and depth of that image (see the demo gif if you haven't looked) Better inpainting model Trained with a stronger NSFW filter on training data. For me the depth-to-image model is a huge highlight and something I wasn't expecting. The NSFW filter is a nothing (it's trivially easy to fine-tune the model on porn if you want, and porn collections are surprisingly easy to come by...). The higher resolution features are interesting. HuggingFace has got the 1.x models working for inference in under 1G of VRAM, and if those optimizations can be preserved it opens up a bunch of interesting possibilities.
- tluyben2 4y ago
  > it's trivially easy to fine-tune the model on porn if you want, and porn collections are surprisingly easy to come by Not really surprised they did this, but be sure some communities will have it fine tuned on porn now-ish. So probably they did it for legal reasons in case illegal materials are generated and they are real companies/people with their names on the release?
  - vintermann 4y ago
    I looked into it (though didn't download the models - too dodgy). One of the NSFW models that's gained traction, and gained attention because it seems to be better at generating even non-porn faces and bodies, is called "Hassan's blend". Hassan mentioned that he'd taken down an earlier checkpoint because it generated undesirable images. Reading between the lines, it likely generated CSAM-like images even without explicit prompting for it.
kmeisthax 4y ago
Is there a good explanation of how to train this from scratch with a custom dataset[0]? I've been looking around the documentation on Huggingface, but all I could find was either how to train unconditional U-Nets[1], or how to use the pretrained Stable Diffusion model to process image prompts (which I already know how to do). Writing a training loop for CLIP manually wound up with me banging against all sorts of strange roadblocks and missing bits of documentation, and I still don't have it working. I'm pretty sure I also need some other trainables at some point, too. [0] Specifically, Wikimedia Commons images in the PD-Art-100 category, because the images will be public domain in the US and the labels CC-BY-SA. This would rule out a lot of the complaints people have about living artists' work getting scraped into the machine; and probably satisfy Debian's ML guidelines. [1] Which actually does work
- echelon 4y ago
  > train this from scratch If you're talking about training from scratch and not fine tuning, that won't be cheap or easy to do. You need thousands upon thousands of dollars of GPU compute [1] and a gigantic data set. I trained something nowhere near the scale of Stable Diffusion on Lambda Labs, and my bill was $14,000. [1] Assuming you rent GPUs hourly, because buying the hardware outright will be prohibitively expensive.
  - pfd1986 4y ago
    Hi, do you have a writeup of that anywhere? Would love to hear (read) more about it
  - kmeisthax 4y ago
    I have... ~11TBs of free disk space and a 1080ti. Obviously nowhere close to being able to crunch all of Wikimedia Commons, but I'm also not trying to beat Stability AI at their own game. I just want to move the arguments people have about art generators beyond "this is unethical copyright laundering" and "the model is taking reference just like a real human".
    - ftufek 4y ago
88stacks 4y ago
Awesome, I’ve put stable diffusion on an api to train a model for anyone to use for free. I’m adding 2.0 to it as we speak! https://88stacks.com https://88stacks.com
- julienmarie 4y ago
  Interesting concept!
- pilaf 4y ago
  > to sue for free Typo of freudian slip? Just kidding of course, nice project!
- dimaor 4y ago
  How is this free? Is it possible to download the checkpoints? I'm asking because I'm running SD locally but my GPU is not good enough to train new checkpoints and while I get the time to work on improve I wanted to use this API in order to generate some models for an illustration book I am working on.
  - 88stacks 4y ago
    It’s free because it’s on my research cluster, not in the cloud and I want to share it. For faster training it will be paid with other features, but to train a basic model will always be free. Im adding download of checkpoints now.
  - nickthegreek 4y ago
    Check out fast-dreambooth colab. I use that to freely train checkpoints that can be downloaded.
- kanonieer 4y ago
  Interesting project but terrible naming.
  - beanjuice 4y ago
    To be clear to the original poster, the naming is terrible because of the nazi associations of the number 88, correct?
lxe 4y ago
I love the graph of GH stars over time. Gives you somewhat of a sense of how fast this space is moving.
dzink 4y ago
The upscaler link on this page https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler https://huggingface.co/stabilityai/stable-diffusion-x4-upsca... is broken. Here is a working link https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler/blob/main/x4-upscaler-ema.ckpt https://huggingface.co/stabilityai/stable-diffusion-x4-upsca...
vsskanth 4y ago
Can this transform an image into a vector illustration ?
- speedgoose 4y ago
  img2img can change the style of an image so you can make it look like a vector illustration but it will still be a bitmap.
- SpliffnCola 4y ago
  It sure can: https://github.com/GeorgLegato/Txt2Vectorgraphics https://github.com/GeorgLegato/Txt2Vectorgraphics It doesn't output color though, only B/W.
1024core 4y ago
What's the bare minimum hardware required to generate images with this model? Can I do something with a 8GB 980? Probably not..? What about CPU only?
- mugivarra69 4y ago
  not that gpu, cpu yes need to wait longer however.
- kiratp 4y ago
  1.4 runs on iPhones. I’m sure we will rapidly see similar for 2.0 https://apps.apple.com/app/id6444050820 https://apps.apple.com/app/id6444050820
charcircuit 4y ago
I dislike how they call their model open source even though there are restrictions on how you can use the model. The ability to use code however you want and not have to worry about if all the code you are using is compatible with your use case is a key part of open source.
- josephcsible 4y ago
  I don't know why you're being downvoted. The model's license is unambiguously noncompliant with the Open Source Definition, yet they falsely claim it to be open source anyway. That's just as misleading as calling a product full of HFCS "sugar free" and saying it's okay because by "sugar", you just mean cane sugar.
- jeroenhd 4y ago
  The code is open source, the model is a data file that the open source code operates on. It's similar to engine recreations for old games (OpenRCT, OpenTTD) that use original, proprietary assets to play the games with their open source engines. Similar to those games, anyone is also able to distribute their own open data files if they so wish It's unlikely anyone actually will start training an open source AI model from scratch because doing so costs insane amounts of money, but the same can be said about the many hours of work recreating game assets can take for open source game engines.
  - charcircuit 4y ago
    I don't know what your point is. They use the terms "open source AI models" and "open source Generative AI models" Yes, someone else could spend the millions of dollars to create a model that actually is open source, but shouldn't the people advertising their models as open source do that?
    - jeroenhd 4y ago
      Should they distribute the data files according to the open source standards? Maybe. "Open" does not mean "open source", though; "open data" does not necessarily allow unlimited access and use of such data available, it's usually behind some kind of ToS document nobody reads and an API key. Applying open source expectations to anything with open in the name will often leave you disappointed outside the FOSS world. Does not openly distributing their data files make their code any less open source? I don't think so. The code is open and licensed with a FOSS license. They spend time and money on creating a model and give the world the ability to replicate their model if it can collect the necessary funds. There are plenty of other open source projects that require vast arrays of server racks and compute power to be useful, that doesn't change anything about the openness of the code.
Sugimot0 4y ago
Amy word on AMD support?
- mugivarra69 4y ago
  just use rocm
  - jacooper 4y ago
    Which has crap support it self.
    - mugivarra69 4y ago
      sure.
- CapsAdmin 4y ago
  The previous version works fine and has performance on par with NVIDIA. I'm on Linux using the ROCm platform. There are some specialized third party performance optimizations you might miss out on though, but nothing major IMO.
techsin101 4y ago
is there a tool that uses to create UX/UI designs. It'd be really cool to get inspirations...
- codsane 4y ago
  I would love that…I’ve heard of this use case for AI described many times but I’ve yet to find anyone doing it! Copilot is great, but for more creative/frontend work it seems like there should be something right? Stable Diffusion is amazing at generating art. Something similar but specialized in UI could be too. Maybe one could make a custom model, but with my lack of design knowledge I’m not even sure where to start… It would surely save my monkey brain from pouring many more hours into looking at existing websites/UI libraries/Dribble and drawing inspiration (copying) from them.
- Godel_unicode 4y ago
  Microsoft Designer is supposedly this, although I’m still waiting on admission to the beta.
fnordpiglet 4y ago
The upscaler is the enhanced zoom and room extrapolation from blade runner. Now that’s cool.
prawn 4y ago
Hopefully related: If I'm a photographer wanting to improve resolution of my content for printing, what's my current best bet for upscaling? Is it realistic to make use of this on the command line, feeding it my own images? Or has someone wrapped it in an app or online service?
- ftufek 4y ago
  You're probably better of using Real-ESRGAN: https://github.com/xinntao/Real-ESRGAN https://github.com/xinntao/Real-ESRGAN. It's pretty solid and fast, even has portable executables you can just use as is. The upscaler that comes with stable diffusion might work for you, but I suspect it'll probably do a better job at upscaling stable diffusion output rather than a natural images (might be wrong though).
  - sendfoods 4y ago
    Not sure about your last claim. The example given in the blog post looks very very close to a natural image.
  - TaylorAlexander 4y ago
    I tried this on one of my home images. I have a nice canon pro 100 printer that can print 13”x19” pictures, and my camera is a 20 megapixel Panasonic GH-5. The printer can print much higher resolution than my camera. So I did take one of my photos and process it with Real-ESRGAN to double the resolution (in each direction, so 4x pixels). The photo is a red barn with redwood trees behind it. It did well increasing the resolution of the barn. It made it look more crisp and bright. But there is an area with some trees in shadow behind the barn, and it lost detail there. Anyway I think it would be fun to play with, just depends on the content of the image and the artists preferences. I still haven’t printed a full page of the upscaled photo but I do want to try that and see how it looks in comparison!
- fassssst 4y ago
  Photoshop and Lightroom have had AI upscaling for awhile.
sorenjan 4y ago
I've seen references to merging models together to be able to generate new kinds of imagery or styles, how does that work? I think you use Dreambooth to make specialized models, and I think I got an idea about how that basically assigns a name to a vector in the latent space that represents the thing you want to generate new imagery of, but can you generate multiple models and blend them together? Edit: Looks like AUTOMATIC1111 can merge three checkpoints. I still don't know how it works technically, but I guess that's how it's done? https://github.com/AUTOMATIC1111/stable-diffusion-webui https://github.com/AUTOMATIC1111/stable-diffusion-webui
- corysama 4y ago
  It’s my understanding that, amazingly enough, blending the models is done by literally performing a trivial linear blend of the raw numbers in the model files. Someone even figured out they could get great compression of specialized model files by first subtracting the base model from the specialized model (using plain arithmetic) before zipping it. Of course, you need the same base file handy when you go to reverse the process.
  - eoerl 4y ago
    It is not typically possible to blend models like that, since the training process is (lateral) order insensitive, as far as the model goes.
    - ShamelessC 4y ago
      Is that still the case when all models have a common ancestor (i.e. finetuned) and haven’t yet overfit on new data?
    - liuliu 4y ago
      I thought so too until found that there are quite a bit of literatures nowadays about "merging" weights, for example, this one: https://arxiv.org/pdf/1811.10515.pdf https://arxiv.org/pdf/1811.10515.pdf and also the OpenCLIP paper.
  - deleted 4y ago
coldblues 4y ago
I just thought about this, so bare in mind that I don't know much of the technical implications of this, but: Couldn't we train a very good model by distributing the dataset along with the computing power using something similar to folding@home?
- ftufek 4y ago
  You would likely be limited by the communication latency between nodes, unless you come up with some unique model architecture or training method. Most of these large scale models are trained on GPUs using very high speed interconnects.
- eugenhotaj 4y ago
  The network communication overhead would be way too high to make this useful. At least for current methods of training large models.
- iosjunkie 4y ago
  stop trying to build skynet
  - Bilal_io 4y ago
    Can't stop something that's already finished. - Skynet
- seydor 4y ago
  i think eventually someone will do
- ArnoVW 4y ago
  The limit for this sort of exercise is "holding everything in memory". Because training of neural networks require that one updates the weights frequently. An NVIDI A100 has a bandwidth of 2 Tb/sec. Your home ADSL something in the order of 10 Mbit. And then there's latency. Mind you, theoretically that is a limitation of our current network architectures. If we could conceive a learning approach that was localised, to the point of being "embarrassingly parallel", perhaps. It would probably be less efficient, but if it is sufficiently parallel to compensate for Amdahl's law, who knows? Less theoretically, one could imagine that we use the same approach that we use in systems engineering in general: functional decomposition. Instead of having one Huge Model To Rule Them All, train separate models that each perform a specific, modular function, and then integrate them. In a sense this is what is currently happening already. Stable Diffusion have one model to generate img2depth, to generate an estimation which parts of a picture are far away from the lense. They have another model to upscale low res images to high res images, etc etc. This is also how the brain works. But it is difficult to see how this sort of approach could be applied to very small scale, low contextual tasks, like folding@home.
acidburnNSA 4y ago
Awesome. I'm installing on Ubuntu 22.04 right now. Ran into a few errors with the default instructions related to CUDA version mismatches with my nvidia driver. Now I'm trying without conda at all. Made a venv. I upgraded to the latest that Ubuntu provides and then downloaded and installed the appropriate CUDA from [1]. That got me farther. Then ran into the fact that the xformers binaries I had in my earlier attempts is now incompatible with my current drivers and CUDA, so rebuiding that one. I'm in the 30-minute compile, but did the `pip install ninja` as recommended by [2] and it's running on a few of my 32 threads now. Ope! Done in 5 mins. Test info from `python -m xformers.info` looks good. Damn still hitting CUDA out of memory issues. I knew I should have bought a bigger GPU back in 2017. Everyone says I have to downgrade pytorch to 1.12.1 for this to not happen. But oh dang that was compiled with a different cuda, oh groan. Maybe I should get conda to work afterall. `torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 5.93 GiB total capacity; 5.62 GiB already allocated; 15.44 MiB free; 5.67 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF` Guess I better go read those docs... to be continued. [1] https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=deb_local https://developer.nvidia.com/cuda-downloads?target_os=Linux&... [2] https://github.com/facebookresearch/xformers https://github.com/facebookresearch/xformers
- jetbooster 4y ago
  Thanks for reminding me why I shouldn't go to my computer right now and try getting this working with my 2070!
- nicoco 4y ago
  Also got this far on my 3080 Ti with the same error message. Oh well, let's wait for the "optimized" forks to pop up.
- Tepix 4y ago
  Which GPU are you using? Used RTX 3090s were relatively cheap in the last couple of weeks...
radu_floricica 4y ago
Speaking of business models for AI, and the fact that stable diffusion is anti-trained for porn. Somebody with an old terra byte image porn collection right now: "Hold my beer, my time has come!"
xzghfat 4y ago
remark
dzink 4y ago
How do you install this version? Do you merge it with latent-diffusion?
imran_khan 4y ago
To put things in perspective, the dataset it's trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs). That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned". But if you want to create custom versions of SD, you can always try out dreambooth: https://github.com/XavierXiao/Dreambooth-Stable-Diffusion https://github.com/XavierXiao/Dreambooth-Stable-Diffusion, that one is actually feasible without spending millions of dollars on GPUs.
imran_khan 4y ago
trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs). That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned". But if you want to create custom versions of SD, you can always try out dreambooth: https://github.com/XavierXiao/Dreambooth-Stable-Diffusion https://github.com/XavierXiao/Dreambooth-Stable-Diffusion, that one is actually feasible without spending millions of dollars on GPUs.
imran-khan 4y ago
To put things in perspective, the dataset it's trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs). That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned". But if you want to create custom versions of SD, you can always try out dreambooth: https://github.com/XavierXiao/Dreambooth-Stable-Diffusion https://github.com/XavierXiao/Dreambooth-Stable-Diffusion, that one is actually feasible without spending millions of dollars on GPUs.
- 2Gkashmiri 4y ago
  hey..... first time i want to dip my toes into this, what graphics card do you suggest?
  - deleted 4y ago
    [deleted]
  - arez 4y ago
    depends on your wallet but RTX 3080 or RTX 3060 are good graphic cards with which you can create these images. If you just want to dip your toes and not spend much you can use Google Colab and rent out googles graphicscards either for free or for $10. Here's a link to a google colab that you can just run for free and that is used a lot https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb https://colab.research.google.com/github/TheLastBen/fast-sta... P.S. if you want to buy a graphics card, make sure to have at least 12GB VRAM
- singularity2001 4y ago
  >> it can't memorize 240TB of data, so it "learned" learning is a form of memorization but yeah
  - visarga 4y ago
    It compresses a whole image down to 1 byte, 60000:1 ratio. That's how much it is allowed to "memorise" from each input on average. Less than a pixel from a whole image.
s1k3s 4y ago
Is there any place where we can learn more about all these AI tools that keep popping up, that is not marketing speak? Also, I see the words 'open' and 'open source' and yet they all require me to sign up to some service, join some beta program, buy credits etc. Are they open source?
- aero-glide2 4y ago
  You can run it locally, you don't need to use their service
- Liquix 4y ago
  Yes, it is open source. Visit the repo linked in the article and follow the setup instructions - no signup required
- this_is_not_you 4y ago
  Did you miss the first part of the article? > It is our pleasure to announce the open-source release of Stable Diffusion Version 2.[0] > The original Stable Diffusion V1 led by CompVis changed the nature of open source AI models and spawned hundreds of other models and innovations all over the world. It had one of the fastest climbs to 10K Github stars of any software, rocketing through 33K stars in less than two months. [0] https://github.com/Stability-AI/stablediffusion https://github.com/Stability-AI/stablediffusion
  - s1k3s 4y ago
    I did. Thank you!
nbzso 4y ago
The crimes against the creative people are getting better and better. What a time to live, when your entire career burns to dust just because. I hope AI gets these programmers jobs soon. Then we all can go to the woods and have a good life, finally.
- rcoveson 4y ago
  You hope that AI gets the jobs of AI programmers soon? I urge you to reconsider the implications of that.
- educaysean 4y ago
  Thanks for your premature concern, but we'll be fine. Despite how it may appear to a layperson such as yourself, the value of human creativity is in no way diminished by the release of this tool or others like it.
  - nbzso 4y ago
    Ok. The irony. Actually, after 20+ years in the tech industry, I will say this: Your beloved corporations don't have a metric called “creativity”, they have a bottom line, and she has all the powers. I am an artist by education and can confirm that creativity is overrated, the processes that artist follows and repetition towards a given goal deliver the results. Whatever feelings or ideas you have, the actual craft is the medium in which you will deliver. Reducing *The Path* to text input is not an artistic or craftsmanship process. There is no creativity involved. May be, someone with more knowledge about the real process and broader visual culture will make more aesthetically right choices. But this can be automated too. This is not a “tool”, like Photoshop. This is something else. And all of you know this. More than 50 percent of frontend code is boilerplate. CRUD apps follow similar logic. Why not automate this repetitive processes first? No. Corporations are starting the automation from the lowest risk crowd—the digital artists, they have low representation, no coherent community and are always ready to sell themselves for pennies. Now they will compete with the machines. And your time in this battle will come. Soon.
NKosmatos 4y ago
Wow, just wow! Newbie question, why can’t someone just take a pre-trained model/network with all the settings/weights/whatever and run it on a different configuration (at a heavily reduced speed)? Isn’t it like a Blender/3D studio/Autocad file, where you can take the original 3D model and then render it using your own hardware? With my single GOU it will take days to raytrace a big scene, whereas someone with multiple higher speced GPUs will need a few minutes.
- ShamelessC 4y ago
  If you use the provided pytorch code, have a modern CPU and enough physical RAM, you can do this currently. As you suggest, inference/generation will take anywhere from hours to days using a CPU instead of a GPU or other ML-accelerator-chip.
- TaylorAlexander 4y ago
  It’s not totally clear what you are asking. The models are trained on something like an NVIDIA A100 which is a super high end machine learning processor, but inference can be run on a home GPU. So this is a “different configuration”. But I think maybe you mean, can they make a model which normally needs a lot of RAM run more slowly on a machine that only has a little RAM? It sounds like there are some tricks to allow the use of smaller amounts of ram by making specific algorithmic tweaks, so if a model normally needs 12GB of VRAM then, depending on the model, it may be possible to modify the algorithm to use 1/2 the RAM for example. But I don’t think it’s the same as other rendering tasks where you can use arbitrarily less compute and just run it longer. Maybe I’m wrong though.
- jeroenhd 4y ago
  The main limitation for running these AIs is that you need tons of VRAM available for your GPU to get any good performance out of them. I don't have a video card with 12GiB of VRAM and I don't know anyone who does. If you're willing to wait more (30 seconds per image, assuming limited image sizes) there are repositories that will run the model on the CPU instead, leveraging your much cheaper RAM. In theory you could swap VRAM in and out in the middle of the rendering process, but this would make the entire process incredibly slow. I think you'll have more success just running the CPU version if you're willing to accept slowdowns.
kennyloginz 4y ago
This reminds me a bit of the rash of Thomas Kinkaid storefronts during the 90s. Nothing personal against the work, I think it’s brilliant, and cheap. Just like a Kinkaid
stavros 4y ago
This is great. I built https://phantasmagoria.me https://phantasmagoria.me because I was excited about this and wanted to make image generation more accessible. I can't wait to see what kinds of images v2 will enable.
Scandiravian 4y ago
What's the potential of using this for image restoration? I've been looking into this recently as I've found a ton of old family photos, that I'd like to digitize and repair some of the damage on them There are a lot of tools available, but I haven't found anything where the result isn't just another kind of bad, so if the upscaling and inference in this model is good, it should in theory be possible to restore images by using the old photos as the seed, right?
- serpix 4y ago
  Current Stable Diffusion's (1.4 and 1.5) img2img can accomplish a lot of this.
myrloc 4y ago
“Adoption” is a generous term to use for a description of Github stars (referring to the first graph). There’s no denying stable diffusion has been gaining popularity, but I think it’s hard to say it’s really being adopted at the same rate it’s getting starred on Github.
- Naracion 4y ago
  FWIW, they only call it "Github stars". The graph that says "adoption" is from a16z [0]. 0: https://a16z.com/2022/11/16/creativity-as-an-app/ https://a16z.com/2022/11/16/creativity-as-an-app/
wigster 4y ago
love the developer adoption graph - so steep i thought they were the y-axis! ;-)
rahul_nyc 4y ago
One thing I’m wondering is what kind of different applications it can be used. Maybe there will be new experiences in the fashion industry like people can train their cloth designs and see how it looks on people. Maybe they don’t need to hire models to do the modelling?
- devxpy 4y ago
  I've also built a similar service[1] that does this with inpainting instead of textual inversion, so it preserves the face exactly and returns in seconds, not hours. [1] - https://app.gooey.ai/FaceInpainting/ https://app.gooey.ai/FaceInpainting/
  - zwirbl 4y ago
    doesn't show any controls after the loading banner in my Firefox
    - deleted 4y ago
      [deleted]
    - devxpy 4y ago
      Hmm, looks like we didn't test on Firefox!
    - devxpy 4y ago
      Fixed. Thanks for reporting. Here's the bug that caused this too - https://bugzilla.mozilla.org/show_bug.cgi?id=1689099 https://bugzilla.mozilla.org/show_bug.cgi?id=1689099
deleted 4y ago
[deleted]
ccbccccbbcccbb 4y ago
stavros 4y ago
It kind of annoys me that they removed NSFW images from the training set. Not because I want to generate porn (though some people do), but because I feel that they're foisting a puritan ethic on me. I don't consider the naked body inherently bad, and I don't like seeing new technology carry this (wrong, in my opinion) stigma. Then again, it's their model, they can do whatever they want with it, but it still leaves me with a weird feeling.
- beernet 4y ago
  Agreed. I could see this being driven by EleutherAI in the background who are very, say, strict when it comes to "alignment".
  - stavros 4y ago
    Hmm, can you elaborate? I don't know much about EleutherAI.
- astrange 4y ago
  It’s annoying when you get NSFW results when you didn’t ask for them, so it may be better to segregate them.
  - stavros 4y ago
    But they aren't segregating them. They didn't release two models, one SFW and one NSFW. They segregated them before, with the filter you could disable, but now it's all SFW-only.
    - astrange 4y ago
      Eh, fine-tuning seems to work well enough that it can be added back in after. Though, previous fine-tunings/textual inversions won’t work since the CLIP encoder has been replaced too. I’d be interested in knowing if it needs to be retrained too for this case.
ghaff 4y ago
I suspect that, if many of the people whining about copyrights in the context of generative AI got their way and made this usage a violation, they wouldn't be happy with the knock-on effects.
tormeh 4y ago
I can't see any progress on AMD/Intel GPU support :( Would love to see Vulkan or at least ROCm support. With SD1 you could follow some guides online to make it work, since PyTorch itself supports ROCm, but the state of non-Nvidia GPU support in the DL space is quite sad.
- magic_at_nodai 4y ago
  Try SHARK on your AMD GPUs for SD. Follow the setup here: https://github.com/nod-ai/SHARK/tree/main/shark/examples/shark_inference/stable_diffusion https://github.com/nod-ai/SHARK/tree/main/shark/examples/sha.... It works with Pytorch -> torch-mlir -> MLIR / IREE -> vulkan. Works on both Windows and Linux. And has a simple gradio web UI https://github.com/nod-ai/SHARK/tree/main/web https://github.com/nod-ai/SHARK/tree/main/web but we plan to enable better UI integrations very soon. Join us on discord https://discord.gg/RUqY2h2s9u https://discord.gg/RUqY2h2s9u if you have any trouble. Appreciate any / all feedback.
in3d 4y ago
In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it should be possible to fine tune this model to create them anyway using DreamBooth or a similar approach, they clearly went for the safe route after taking some heat. 1. https://twitter.com/emostaque/status/1595731407095140352?s=46&t=5Zxp66AzZrIrvccjqNMJPA https://twitter.com/emostaque/status/1595731407095140352?s=4...
- poisonarena 4y ago
  this is really disappointing
- arez 4y ago
  does this mean that stuff like artstation and deviantart doesn't work anymore as prompts? That would be a huge change
  - in3d 4y ago
    Based on sample images I’ve seen, “Greg Rutkowski” doesn’t work anymore for example.
  - siraben 4y ago
    Ran the model locally. Neither "trending on artstation" nor "Greg Rutowski" make any differences to the image anymore.[0] I suspect that people will find keywords that would improve the aesthetics further again, or that fine-tuning will also take place. [0] https://imgsli.com/MTM1ODQ5 https://imgsli.com/MTM1ODQ5
    - astrange 4y ago
      “Prompting” and “keywords” are not an essential part of this technology. If you like tokens, make your own tokens with textual-inversion or image inputs.
- CuriouslyC 4y ago
  Removing NSFW content is fine, people who care about that can work around it easily. Removing celebrities and commercial artists was a mistake though and I expect this will need to be really impressive in other ways or people aren't going to bother using it.
spcebar 4y ago
Well darn. This is an awesome leap, but I've spent the last few months making a card game using Stable Diffusion art and I guess now I need to go back and go over everything again. Congratulations to the SD team on another wonderful step forward!
mensetmanusman 4y ago
Highly recommend downloading draw anything on iOS, sd1.4 is available and it is quite fun!
devxpy 4y ago
Try it out here - https://app.gooey.ai/CompareText2Img/?example_id=1uONp1IBt0Y https://app.gooey.ai/CompareText2Img/?example_id=1uONp1IBt0Y
sssswww 4y ago
galactic freaks
sawirricardo 4y ago
Here are some which you'll encounter if you want to rabbit hole this trend - replicate.com - banana.dev - huggingface.co - lambdalabs.com - astriaAI - lexica.art for prompt inspiration

1368 points by hardmaru 4y ago

There's nothing in the release notes that says whether 2.0 can do hands without a 99% chance of producing deformed results.

minimaxir 4y ago

GitHub Repo: https://github.com/Stability-AI/stablediffusion https://github.com/Stability-AI/stablediffusion HuggingFace Space (currently overloaded unsurprisingly): https://huggingface.co/spaces/stabilityai/stable-diffusion https://huggingface.co/spaces/stabilityai/stable-diffusion Doing a 2.0 release on a (US) 2-day holiday weekend is an interesting move. It seems a tad more difficult to set up the model than the previous version.

gst 4y ago
> Doing a 2.0 release on a (US) 2-day holiday weekend is an interesting move. Their HQ seems to be located in London.
- minimaxir 4y ago
  True, but it's going to dampen the launch a bit.
  - jehb 4y ago
    Is it? For me, I'm in tech but nowhere near anything for which playing with a new SD release would be relevant to my day job. Having a couple of extra days off to play with a new tech toy probably means I'll use it more.
  - smcleod 4y ago
    I don't see why? The world is a whole lot bigger than the USA!
Operyl 4y ago
Seems like a potentially good time to launch it, lots of young people with free time.
- Gigachad 4y ago
  Less likely journalist whingers will pick it up too.
- kennyloginz 4y ago
  Definitely something to talk about or share with the fam.
Existenceblinks 4y ago
I wonder why these AI repos' documentation are so bad compared to what we are used to in general. Where is intro/get started/example(commands)/config(docs) etc.
- blihp 4y ago
  I believe it's a combination of the fact that most of these models are basically 'research dumps' primarily targeting other researchers and given this they are assuming a level of familiarity with related tools/libraries. So it's up to interested people in the community to take it the last mile/block/whatever to make it easy to use, address specific use cases etc. for use by a less academic/technical audience.
  - Existenceblinks 4y ago
    Yeah, generally speaking, it's not optimized for open collaboration (not even close .. pun intended).
IanCal 4y ago
I think it's looking fairly similar, the first one was a bit tricky too. Later improvements by the community made it clearer. The docs aren't good though, it tells you to download two things when actually I think you only need one. If you do need two then it doesn't tell you at all where to put the second. You really need xformers if you're doing it at home, I've got a 3090 and it blew through the ram without it. However, the instructions didn't work for me for compiling and there's an incompatibility if you try and install from conda. You can have it work but you need to upgrade python from 3.8.5 to 3.9 in the yaml file first, then you can install it (xformers needs 3.9+, and something else in SD breaks on 3.10+ so 3.9 works). This needs the classic "sit next to a new person installing it by following the docs and see what problems they hit, fix the docs and start from scratch again" process. Looks good, though so far the images I've made don't look as nice as with 1.4, but I guess that's largely down to finding the right tweaks for the model and right magic wording for the prompts.

rogers18445 4y ago

They apparently tried to combat NSFW generation by filtering the training dataset not to include any.

throwup 4y ago
You can generate all the bloody violent gore you like, but god forbid anybody see a human body in its natural state
- echelon 4y ago
  There is worry about generating illegal content. If the model understands multiple concepts, it can combine them.
- dang 4y ago
  "Avoid generic tangents." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
argella 4y ago
Bummer. AI porn is fun.
- deleted 4y ago
  [deleted]
- Gigachad 4y ago
  The future is probably models trained almost exclusively on porn.
  - joshspankit 4y ago
    Porn has driven many tech advances. I predict that models trained on specific porn genres will appear as soon as training a good model is doable for under $5000. They’ll get here much quicker if we get video to that mark first.

satvikpendem 4y ago

Looks good. I've gotten bored with AI image generation these days however, after using a lot of SD the past few months. I suppose that's the hedonic treadmill in action.

quickthrower2 4y ago
Another thing to take for granted eh!

natch 4y ago

> greatly improves the quality of the generated images compared to earlier V1 releases That’s great!

minimaxir 4y ago
The repo provides a chart with a quantitative measure (FID/CLIP score) where the new 2.0 models do indeed have much better results than the earlier 1.5 model.
- natch 4y ago
  Nice, will be interesting to see.

theylovezmw 4y ago

Interesting this is being marketed as a 2.0 release so quickly after the first version was launched. These new updates are quite great but are they so game-changing that it is considered 2.0?

vanshg 4y ago
It doesn't need to be game-chaning to be considered V2. It just needs to be "the next version"

LASR 4y ago

I am a solo dev working on a creative content creation app to leverage the latest developments in AI. Demoing even the v1 of stable diffusion to the non-technical general users blows them away completely. Now that v2 is here, it’s clear we’re not able to keep pace in developing products to take advantage of it. The general public still is blown away by autosuggest in mobile OS keyboards. Very few really know how far AI tech has evolved. Huge market opportunity for folks wanting to ride the wave here. This is exciting for me personally, since I can keep plugging in newer and better versions of these models into my app and it becomes better. Even some of the tech folks I demo my app to, are simply amazed how I can manage to do this solo.

jnrk 4y ago
> The general public still is blown away by autosuggest in mobile OS keyboards. And it's still not available in my language on iOS... :( (Norwegian)
ksec 4y ago
Last time we had people in the field of AI writing on HN expecting another AI Winter. I thought all the improvement we have now, and we might get in the future like Stable Diffusion 3.0 and possibly many others, while not L5 Self Driving Car or General AI, has yet to be fully used outside of Tech community, or even in Tech itself. The long tail of distribution and positive feedback loop will sustain its development for another 10 years.
searchableguy 4y ago
What are you building?
- KidComputer 4y ago
  - wfme 4y ago
    This comment pretty clearly breaks the commenting guidelines https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html >Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Please don't fulminate. Please don't sneer, including at the rest of the community. Edit out swipes. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.

knicholes 4y ago

Does anyone have a good source for all sorts of prompts for image generation?

gedy 4y ago
https://lexica.art https://lexica.art
mromanuk 4y ago
Have you seen lexica.art? There are a few others, but I don’t remember the URLs
asicsp 4y ago
https://publicprompts.art/ https://publicprompts.art/ is amazing
jasfi 4y ago
InventAI: https://inventai.xyz https://inventai.xyz is being developed, will help a lot with prompt engineering.

WatchDog 4y ago

Are the GPU memory requirements different for this release? Is now it possible to generate higher resolution images with less memory?

mugivarra69 4y ago
dont thinkso, ran on my macbook pro, same footprint. idk cuda side.
- swyx 4y ago
  how do you check footprint?

wyldfire 4y ago

depth2img looks really interesting. I was thinking that someone should train an art model like SD on 3d models+textures. This isn't quite that but it seems like it gets some of that effect.

wyldfire 4y ago

This one's publicly downloadable? I think I must've missed 1.5. it had been postponed for a while (for good reasons discussed throughout threads here) and I didn't notice whether it had been released.

minimaxir 4y ago
1.5 was released, although it was not dramatically better than 1.4 (outside of better inpainting) so it didn't get much buzz. https://huggingface.co/runwayml/stable-diffusion-v1-5 https://huggingface.co/runwayml/stable-diffusion-v1-5

liuliu 4y ago

Seems the structure of UNet hasn't changed other than the text encoder input (768 to 1024). The biggest change is on the text encoder, switched from ViT-L14 to ViT-H14 and fine-tuned based on https://arxiv.org/pdf/2109.01903.pdf https://arxiv.org/pdf/2109.01903.pdf. Seems the 768-v model, if used properly, can substantially speed-up the generation, but not exactly sure yet. Seems straightforward to switch to 512-base model for my app next week.

millimeterman 4y ago
I'm disappointed they didn't push parameter count higher, but I suppose they want to maintain the ability to run on older/lower end consumer GPUs. Unfortunately it severely limits how high-quality the output can be.
- ShamelessC 4y ago
  Can’t forget time it takes to run inference, even on the latest A100/H100. Generating in under e.g. ten seconds enables more use cases (and so on until high fps video is possible).
- naillo 4y ago
  They're motivating that choice via this paper: https://arxiv.org/pdf/2203.15556.pdf https://arxiv.org/pdf/2203.15556.pdf The paper shows that you can get better performance than gpt-3 with a much smaller model if you bump up the training time and training data like x4.
  - macrolime 4y ago
    Larger models are still much better. Google's parti model can do text perfectly and follows prompts way more accurately than Stable Diffusion. It's 20B parameters and with the latest int8 optimizations it should be possible to get that running on a consumer 24GB card in theory. I think they're looking into larger models later though
smeagull 4y ago
Oh shit, I think that means it's CPU only for me now.

dzink 4y ago

This Thanksgiving, I would like to extend a very warm Thank You to the Stable Diffusion team! Side note: The 4x upscaler model is showing as unavailable if you follow the hugging face link to it.

nl 4y ago

Highlights: 768x768 native models (v1.x maxed out at 512x512) a built-in 4x upscaler: "Combined with our text-to-image models, Stable Diffusion 2.0 can now generate images with resolutions of 2048x2048–or even higher." Depth-to-Image Diffusion Model: "infers the depth of an input image, and then generates new images using both the text and depth information." Depth-to-Image can offer all sorts of new creative applications, delivering transformations that look radically different from the original but which still preserve the coherence and depth of that image (see the demo gif if you haven't looked) Better inpainting model Trained with a stronger NSFW filter on training data. For me the depth-to-image model is a huge highlight and something I wasn't expecting. The NSFW filter is a nothing (it's trivially easy to fine-tune the model on porn if you want, and porn collections are surprisingly easy to come by...). The higher resolution features are interesting. HuggingFace has got the 1.x models working for inference in under 1G of VRAM, and if those optimizations can be preserved it opens up a bunch of interesting possibilities.

tluyben2 4y ago
> it's trivially easy to fine-tune the model on porn if you want, and porn collections are surprisingly easy to come by Not really surprised they did this, but be sure some communities will have it fine tuned on porn now-ish. So probably they did it for legal reasons in case illegal materials are generated and they are real companies/people with their names on the release?
- vintermann 4y ago
  I looked into it (though didn't download the models - too dodgy). One of the NSFW models that's gained traction, and gained attention because it seems to be better at generating even non-porn faces and bodies, is called "Hassan's blend". Hassan mentioned that he'd taken down an earlier checkpoint because it generated undesirable images. Reading between the lines, it likely generated CSAM-like images even without explicit prompting for it.

kmeisthax 4y ago

Is there a good explanation of how to train this from scratch with a custom dataset[0]? I've been looking around the documentation on Huggingface, but all I could find was either how to train unconditional U-Nets[1], or how to use the pretrained Stable Diffusion model to process image prompts (which I already know how to do). Writing a training loop for CLIP manually wound up with me banging against all sorts of strange roadblocks and missing bits of documentation, and I still don't have it working. I'm pretty sure I also need some other trainables at some point, too. [0] Specifically, Wikimedia Commons images in the PD-Art-100 category, because the images will be public domain in the US and the labels CC-BY-SA. This would rule out a lot of the complaints people have about living artists' work getting scraped into the machine; and probably satisfy Debian's ML guidelines. [1] Which actually does work

echelon 4y ago
> train this from scratch If you're talking about training from scratch and not fine tuning, that won't be cheap or easy to do. You need thousands upon thousands of dollars of GPU compute [1] and a gigantic data set. I trained something nowhere near the scale of Stable Diffusion on Lambda Labs, and my bill was $14,000. [1] Assuming you rent GPUs hourly, because buying the hardware outright will be prohibitively expensive.
- pfd1986 4y ago
  Hi, do you have a writeup of that anywhere? Would love to hear (read) more about it
- kmeisthax 4y ago
  I have... ~11TBs of free disk space and a 1080ti. Obviously nowhere close to being able to crunch all of Wikimedia Commons, but I'm also not trying to beat Stability AI at their own game. I just want to move the arguments people have about art generators beyond "this is unethical copyright laundering" and "the model is taking reference just like a real human".
  - ftufek 4y ago

88stacks 4y ago

Awesome, I’ve put stable diffusion on an api to train a model for anyone to use for free. I’m adding 2.0 to it as we speak! https://88stacks.com https://88stacks.com

julienmarie 4y ago
Interesting concept!
pilaf 4y ago
> to sue for free Typo of freudian slip? Just kidding of course, nice project!
dimaor 4y ago
How is this free? Is it possible to download the checkpoints? I'm asking because I'm running SD locally but my GPU is not good enough to train new checkpoints and while I get the time to work on improve I wanted to use this API in order to generate some models for an illustration book I am working on.
- 88stacks 4y ago
  It’s free because it’s on my research cluster, not in the cloud and I want to share it. For faster training it will be paid with other features, but to train a basic model will always be free. Im adding download of checkpoints now.
- nickthegreek 4y ago
  Check out fast-dreambooth colab. I use that to freely train checkpoints that can be downloaded.
kanonieer 4y ago
Interesting project but terrible naming.
- beanjuice 4y ago
  To be clear to the original poster, the naming is terrible because of the nazi associations of the number 88, correct?

lxe 4y ago

I love the graph of GH stars over time. Gives you somewhat of a sense of how fast this space is moving.

dzink 4y ago

The upscaler link on this page https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler https://huggingface.co/stabilityai/stable-diffusion-x4-upsca... is broken. Here is a working link https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler/blob/main/x4-upscaler-ema.ckpt https://huggingface.co/stabilityai/stable-diffusion-x4-upsca...

vsskanth 4y ago

Can this transform an image into a vector illustration ?

speedgoose 4y ago
img2img can change the style of an image so you can make it look like a vector illustration but it will still be a bitmap.
SpliffnCola 4y ago
It sure can: https://github.com/GeorgLegato/Txt2Vectorgraphics https://github.com/GeorgLegato/Txt2Vectorgraphics It doesn't output color though, only B/W.

1024core 4y ago

What's the bare minimum hardware required to generate images with this model? Can I do something with a 8GB 980? Probably not..? What about CPU only?

mugivarra69 4y ago
not that gpu, cpu yes need to wait longer however.
kiratp 4y ago
1.4 runs on iPhones. I’m sure we will rapidly see similar for 2.0 https://apps.apple.com/app/id6444050820 https://apps.apple.com/app/id6444050820

charcircuit 4y ago

I dislike how they call their model open source even though there are restrictions on how you can use the model. The ability to use code however you want and not have to worry about if all the code you are using is compatible with your use case is a key part of open source.

josephcsible 4y ago
I don't know why you're being downvoted. The model's license is unambiguously noncompliant with the Open Source Definition, yet they falsely claim it to be open source anyway. That's just as misleading as calling a product full of HFCS "sugar free" and saying it's okay because by "sugar", you just mean cane sugar.
jeroenhd 4y ago
The code is open source, the model is a data file that the open source code operates on. It's similar to engine recreations for old games (OpenRCT, OpenTTD) that use original, proprietary assets to play the games with their open source engines. Similar to those games, anyone is also able to distribute their own open data files if they so wish It's unlikely anyone actually will start training an open source AI model from scratch because doing so costs insane amounts of money, but the same can be said about the many hours of work recreating game assets can take for open source game engines.
- charcircuit 4y ago
  I don't know what your point is. They use the terms "open source AI models" and "open source Generative AI models" Yes, someone else could spend the millions of dollars to create a model that actually is open source, but shouldn't the people advertising their models as open source do that?
  - jeroenhd 4y ago
    Should they distribute the data files according to the open source standards? Maybe. "Open" does not mean "open source", though; "open data" does not necessarily allow unlimited access and use of such data available, it's usually behind some kind of ToS document nobody reads and an API key. Applying open source expectations to anything with open in the name will often leave you disappointed outside the FOSS world. Does not openly distributing their data files make their code any less open source? I don't think so. The code is open and licensed with a FOSS license. They spend time and money on creating a model and give the world the ability to replicate their model if it can collect the necessary funds. There are plenty of other open source projects that require vast arrays of server racks and compute power to be useful, that doesn't change anything about the openness of the code.

Sugimot0 4y ago

Amy word on AMD support?

mugivarra69 4y ago
just use rocm
- jacooper 4y ago
  Which has crap support it self.
  - mugivarra69 4y ago
    sure.
CapsAdmin 4y ago
The previous version works fine and has performance on par with NVIDIA. I'm on Linux using the ROCm platform. There are some specialized third party performance optimizations you might miss out on though, but nothing major IMO.

techsin101 4y ago

is there a tool that uses to create UX/UI designs. It'd be really cool to get inspirations...

codsane 4y ago
I would love that…I’ve heard of this use case for AI described many times but I’ve yet to find anyone doing it! Copilot is great, but for more creative/frontend work it seems like there should be something right? Stable Diffusion is amazing at generating art. Something similar but specialized in UI could be too. Maybe one could make a custom model, but with my lack of design knowledge I’m not even sure where to start… It would surely save my monkey brain from pouring many more hours into looking at existing websites/UI libraries/Dribble and drawing inspiration (copying) from them.
Godel_unicode 4y ago
Microsoft Designer is supposedly this, although I’m still waiting on admission to the beta.

fnordpiglet 4y ago

The upscaler is the enhanced zoom and room extrapolation from blade runner. Now that’s cool.

prawn 4y ago

Hopefully related: If I'm a photographer wanting to improve resolution of my content for printing, what's my current best bet for upscaling? Is it realistic to make use of this on the command line, feeding it my own images? Or has someone wrapped it in an app or online service?

ftufek 4y ago
You're probably better of using Real-ESRGAN: https://github.com/xinntao/Real-ESRGAN https://github.com/xinntao/Real-ESRGAN. It's pretty solid and fast, even has portable executables you can just use as is. The upscaler that comes with stable diffusion might work for you, but I suspect it'll probably do a better job at upscaling stable diffusion output rather than a natural images (might be wrong though).
- sendfoods 4y ago
  Not sure about your last claim. The example given in the blog post looks very very close to a natural image.
- TaylorAlexander 4y ago
  I tried this on one of my home images. I have a nice canon pro 100 printer that can print 13”x19” pictures, and my camera is a 20 megapixel Panasonic GH-5. The printer can print much higher resolution than my camera. So I did take one of my photos and process it with Real-ESRGAN to double the resolution (in each direction, so 4x pixels). The photo is a red barn with redwood trees behind it. It did well increasing the resolution of the barn. It made it look more crisp and bright. But there is an area with some trees in shadow behind the barn, and it lost detail there. Anyway I think it would be fun to play with, just depends on the content of the image and the artists preferences. I still haven’t printed a full page of the upscaled photo but I do want to try that and see how it looks in comparison!
fassssst 4y ago
Photoshop and Lightroom have had AI upscaling for awhile.

sorenjan 4y ago

I've seen references to merging models together to be able to generate new kinds of imagery or styles, how does that work? I think you use Dreambooth to make specialized models, and I think I got an idea about how that basically assigns a name to a vector in the latent space that represents the thing you want to generate new imagery of, but can you generate multiple models and blend them together? Edit: Looks like AUTOMATIC1111 can merge three checkpoints. I still don't know how it works technically, but I guess that's how it's done? https://github.com/AUTOMATIC1111/stable-diffusion-webui https://github.com/AUTOMATIC1111/stable-diffusion-webui

corysama 4y ago
It’s my understanding that, amazingly enough, blending the models is done by literally performing a trivial linear blend of the raw numbers in the model files. Someone even figured out they could get great compression of specialized model files by first subtracting the base model from the specialized model (using plain arithmetic) before zipping it. Of course, you need the same base file handy when you go to reverse the process.
- eoerl 4y ago
  It is not typically possible to blend models like that, since the training process is (lateral) order insensitive, as far as the model goes.
  - ShamelessC 4y ago
    Is that still the case when all models have a common ancestor (i.e. finetuned) and haven’t yet overfit on new data?
  - liuliu 4y ago
    I thought so too until found that there are quite a bit of literatures nowadays about "merging" weights, for example, this one: https://arxiv.org/pdf/1811.10515.pdf https://arxiv.org/pdf/1811.10515.pdf and also the OpenCLIP paper.
- deleted 4y ago

coldblues 4y ago

I just thought about this, so bare in mind that I don't know much of the technical implications of this, but: Couldn't we train a very good model by distributing the dataset along with the computing power using something similar to folding@home?

ftufek 4y ago
You would likely be limited by the communication latency between nodes, unless you come up with some unique model architecture or training method. Most of these large scale models are trained on GPUs using very high speed interconnects.
eugenhotaj 4y ago
The network communication overhead would be way too high to make this useful. At least for current methods of training large models.
iosjunkie 4y ago
stop trying to build skynet
- Bilal_io 4y ago
  Can't stop something that's already finished. - Skynet
seydor 4y ago
i think eventually someone will do
ArnoVW 4y ago
The limit for this sort of exercise is "holding everything in memory". Because training of neural networks require that one updates the weights frequently. An NVIDI A100 has a bandwidth of 2 Tb/sec. Your home ADSL something in the order of 10 Mbit. And then there's latency. Mind you, theoretically that is a limitation of our current network architectures. If we could conceive a learning approach that was localised, to the point of being "embarrassingly parallel", perhaps. It would probably be less efficient, but if it is sufficiently parallel to compensate for Amdahl's law, who knows? Less theoretically, one could imagine that we use the same approach that we use in systems engineering in general: functional decomposition. Instead of having one Huge Model To Rule Them All, train separate models that each perform a specific, modular function, and then integrate them. In a sense this is what is currently happening already. Stable Diffusion have one model to generate img2depth, to generate an estimation which parts of a picture are far away from the lense. They have another model to upscale low res images to high res images, etc etc. This is also how the brain works. But it is difficult to see how this sort of approach could be applied to very small scale, low contextual tasks, like folding@home.

acidburnNSA 4y ago

Awesome. I'm installing on Ubuntu 22.04 right now. Ran into a few errors with the default instructions related to CUDA version mismatches with my nvidia driver. Now I'm trying without conda at all. Made a venv. I upgraded to the latest that Ubuntu provides and then downloaded and installed the appropriate CUDA from [1]. That got me farther. Then ran into the fact that the xformers binaries I had in my earlier attempts is now incompatible with my current drivers and CUDA, so rebuiding that one. I'm in the 30-minute compile, but did the `pip install ninja` as recommended by [2] and it's running on a few of my 32 threads now. Ope! Done in 5 mins. Test info from `python -m xformers.info` looks good. Damn still hitting CUDA out of memory issues. I knew I should have bought a bigger GPU back in 2017. Everyone says I have to downgrade pytorch to 1.12.1 for this to not happen. But oh dang that was compiled with a different cuda, oh groan. Maybe I should get conda to work afterall. `torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 5.93 GiB total capacity; 5.62 GiB already allocated; 15.44 MiB free; 5.67 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF` Guess I better go read those docs... to be continued. [1] https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=deb_local https://developer.nvidia.com/cuda-downloads?target_os=Linux&... [2] https://github.com/facebookresearch/xformers https://github.com/facebookresearch/xformers

jetbooster 4y ago
Thanks for reminding me why I shouldn't go to my computer right now and try getting this working with my 2070!
nicoco 4y ago
Also got this far on my 3080 Ti with the same error message. Oh well, let's wait for the "optimized" forks to pop up.
Tepix 4y ago
Which GPU are you using? Used RTX 3090s were relatively cheap in the last couple of weeks...

radu_floricica 4y ago

Speaking of business models for AI, and the fact that stable diffusion is anti-trained for porn. Somebody with an old terra byte image porn collection right now: "Hold my beer, my time has come!"

xzghfat 4y ago

remark

dzink 4y ago

How do you install this version? Do you merge it with latent-diffusion?

imran_khan 4y ago

To put things in perspective, the dataset it's trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs). That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned". But if you want to create custom versions of SD, you can always try out dreambooth: https://github.com/XavierXiao/Dreambooth-Stable-Diffusion https://github.com/XavierXiao/Dreambooth-Stable-Diffusion, that one is actually feasible without spending millions of dollars on GPUs.

imran_khan 4y ago

trained on is ~240TB and Stability has over ~4000 Nvidia A100 (which is much faster than a 1080ti). Without those ingredients, you're highly unlikely to get a model that's worth using (it'll produce mostly useless outputs). That argument also makes little sense when you consider that the model is a couple gigabytes itself, it can't memorize 240TB of data, so it "learned". But if you want to create custom versions of SD, you can always try out dreambooth: https://github.com/XavierXiao/Dreambooth-Stable-Diffusion https://github.com/XavierXiao/Dreambooth-Stable-Diffusion, that one is actually feasible without spending millions of dollars on GPUs.

imran-khan 4y ago

2Gkashmiri 4y ago
hey..... first time i want to dip my toes into this, what graphics card do you suggest?
- deleted 4y ago
  [deleted]
- arez 4y ago
  depends on your wallet but RTX 3080 or RTX 3060 are good graphic cards with which you can create these images. If you just want to dip your toes and not spend much you can use Google Colab and rent out googles graphicscards either for free or for $10. Here's a link to a google colab that you can just run for free and that is used a lot https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb https://colab.research.google.com/github/TheLastBen/fast-sta... P.S. if you want to buy a graphics card, make sure to have at least 12GB VRAM
singularity2001 4y ago
>> it can't memorize 240TB of data, so it "learned" learning is a form of memorization but yeah
- visarga 4y ago
  It compresses a whole image down to 1 byte, 60000:1 ratio. That's how much it is allowed to "memorise" from each input on average. Less than a pixel from a whole image.

s1k3s 4y ago

Is there any place where we can learn more about all these AI tools that keep popping up, that is not marketing speak? Also, I see the words 'open' and 'open source' and yet they all require me to sign up to some service, join some beta program, buy credits etc. Are they open source?

aero-glide2 4y ago
You can run it locally, you don't need to use their service
Liquix 4y ago
Yes, it is open source. Visit the repo linked in the article and follow the setup instructions - no signup required
this_is_not_you 4y ago
Did you miss the first part of the article? > It is our pleasure to announce the open-source release of Stable Diffusion Version 2.[0] > The original Stable Diffusion V1 led by CompVis changed the nature of open source AI models and spawned hundreds of other models and innovations all over the world. It had one of the fastest climbs to 10K Github stars of any software, rocketing through 33K stars in less than two months. [0] https://github.com/Stability-AI/stablediffusion https://github.com/Stability-AI/stablediffusion
- s1k3s 4y ago
  I did. Thank you!

nbzso 4y ago

The crimes against the creative people are getting better and better. What a time to live, when your entire career burns to dust just because. I hope AI gets these programmers jobs soon. Then we all can go to the woods and have a good life, finally.

rcoveson 4y ago
You hope that AI gets the jobs of AI programmers soon? I urge you to reconsider the implications of that.
educaysean 4y ago
Thanks for your premature concern, but we'll be fine. Despite how it may appear to a layperson such as yourself, the value of human creativity is in no way diminished by the release of this tool or others like it.
- nbzso 4y ago
  Ok. The irony. Actually, after 20+ years in the tech industry, I will say this: Your beloved corporations don't have a metric called “creativity”, they have a bottom line, and she has all the powers. I am an artist by education and can confirm that creativity is overrated, the processes that artist follows and repetition towards a given goal deliver the results. Whatever feelings or ideas you have, the actual craft is the medium in which you will deliver. Reducing *The Path* to text input is not an artistic or craftsmanship process. There is no creativity involved. May be, someone with more knowledge about the real process and broader visual culture will make more aesthetically right choices. But this can be automated too. This is not a “tool”, like Photoshop. This is something else. And all of you know this. More than 50 percent of frontend code is boilerplate. CRUD apps follow similar logic. Why not automate this repetitive processes first? No. Corporations are starting the automation from the lowest risk crowd—the digital artists, they have low representation, no coherent community and are always ready to sell themselves for pennies. Now they will compete with the machines. And your time in this battle will come. Soon.

NKosmatos 4y ago

Wow, just wow! Newbie question, why can’t someone just take a pre-trained model/network with all the settings/weights/whatever and run it on a different configuration (at a heavily reduced speed)? Isn’t it like a Blender/3D studio/Autocad file, where you can take the original 3D model and then render it using your own hardware? With my single GOU it will take days to raytrace a big scene, whereas someone with multiple higher speced GPUs will need a few minutes.

ShamelessC 4y ago
If you use the provided pytorch code, have a modern CPU and enough physical RAM, you can do this currently. As you suggest, inference/generation will take anywhere from hours to days using a CPU instead of a GPU or other ML-accelerator-chip.
TaylorAlexander 4y ago
It’s not totally clear what you are asking. The models are trained on something like an NVIDIA A100 which is a super high end machine learning processor, but inference can be run on a home GPU. So this is a “different configuration”. But I think maybe you mean, can they make a model which normally needs a lot of RAM run more slowly on a machine that only has a little RAM? It sounds like there are some tricks to allow the use of smaller amounts of ram by making specific algorithmic tweaks, so if a model normally needs 12GB of VRAM then, depending on the model, it may be possible to modify the algorithm to use 1/2 the RAM for example. But I don’t think it’s the same as other rendering tasks where you can use arbitrarily less compute and just run it longer. Maybe I’m wrong though.
jeroenhd 4y ago
The main limitation for running these AIs is that you need tons of VRAM available for your GPU to get any good performance out of them. I don't have a video card with 12GiB of VRAM and I don't know anyone who does. If you're willing to wait more (30 seconds per image, assuming limited image sizes) there are repositories that will run the model on the CPU instead, leveraging your much cheaper RAM. In theory you could swap VRAM in and out in the middle of the rendering process, but this would make the entire process incredibly slow. I think you'll have more success just running the CPU version if you're willing to accept slowdowns.

kennyloginz 4y ago

This reminds me a bit of the rash of Thomas Kinkaid storefronts during the 90s. Nothing personal against the work, I think it’s brilliant, and cheap. Just like a Kinkaid

stavros 4y ago

This is great. I built https://phantasmagoria.me https://phantasmagoria.me because I was excited about this and wanted to make image generation more accessible. I can't wait to see what kinds of images v2 will enable.

Scandiravian 4y ago

What's the potential of using this for image restoration? I've been looking into this recently as I've found a ton of old family photos, that I'd like to digitize and repair some of the damage on them There are a lot of tools available, but I haven't found anything where the result isn't just another kind of bad, so if the upscaling and inference in this model is good, it should in theory be possible to restore images by using the old photos as the seed, right?

serpix 4y ago
Current Stable Diffusion's (1.4 and 1.5) img2img can accomplish a lot of this.

myrloc 4y ago

“Adoption” is a generous term to use for a description of Github stars (referring to the first graph). There’s no denying stable diffusion has been gaining popularity, but I think it’s hard to say it’s really being adopted at the same rate it’s getting starred on Github.

Naracion 4y ago
FWIW, they only call it "Github stars". The graph that says "adoption" is from a16z [0]. 0: https://a16z.com/2022/11/16/creativity-as-an-app/ https://a16z.com/2022/11/16/creativity-as-an-app/

wigster 4y ago

love the developer adoption graph - so steep i thought they were the y-axis! ;-)

rahul_nyc 4y ago

One thing I’m wondering is what kind of different applications it can be used. Maybe there will be new experiences in the fashion industry like people can train their cloth designs and see how it looks on people. Maybe they don’t need to hire models to do the modelling?

devxpy 4y ago
I've also built a similar service[1] that does this with inpainting instead of textual inversion, so it preserves the face exactly and returns in seconds, not hours. [1] - https://app.gooey.ai/FaceInpainting/ https://app.gooey.ai/FaceInpainting/
- zwirbl 4y ago
  doesn't show any controls after the loading banner in my Firefox
  - deleted 4y ago
    [deleted]
  - devxpy 4y ago
    Hmm, looks like we didn't test on Firefox!
  - devxpy 4y ago
    Fixed. Thanks for reporting. Here's the bug that caused this too - https://bugzilla.mozilla.org/show_bug.cgi?id=1689099 https://bugzilla.mozilla.org/show_bug.cgi?id=1689099

deleted 4y ago

[deleted]

ccbccccbbcccbb 4y ago

stavros 4y ago

It kind of annoys me that they removed NSFW images from the training set. Not because I want to generate porn (though some people do), but because I feel that they're foisting a puritan ethic on me. I don't consider the naked body inherently bad, and I don't like seeing new technology carry this (wrong, in my opinion) stigma. Then again, it's their model, they can do whatever they want with it, but it still leaves me with a weird feeling.

beernet 4y ago
Agreed. I could see this being driven by EleutherAI in the background who are very, say, strict when it comes to "alignment".
- stavros 4y ago
  Hmm, can you elaborate? I don't know much about EleutherAI.
astrange 4y ago
It’s annoying when you get NSFW results when you didn’t ask for them, so it may be better to segregate them.
- stavros 4y ago
  But they aren't segregating them. They didn't release two models, one SFW and one NSFW. They segregated them before, with the filter you could disable, but now it's all SFW-only.
  - astrange 4y ago
    Eh, fine-tuning seems to work well enough that it can be added back in after. Though, previous fine-tunings/textual inversions won’t work since the CLIP encoder has been replaced too. I’d be interested in knowing if it needs to be retrained too for this case.

ghaff 4y ago

I suspect that, if many of the people whining about copyrights in the context of generative AI got their way and made this usage a violation, they wouldn't be happy with the knock-on effects.

tormeh 4y ago

I can't see any progress on AMD/Intel GPU support :( Would love to see Vulkan or at least ROCm support. With SD1 you could follow some guides online to make it work, since PyTorch itself supports ROCm, but the state of non-Nvidia GPU support in the DL space is quite sad.

magic_at_nodai 4y ago
Try SHARK on your AMD GPUs for SD. Follow the setup here: https://github.com/nod-ai/SHARK/tree/main/shark/examples/shark_inference/stable_diffusion https://github.com/nod-ai/SHARK/tree/main/shark/examples/sha.... It works with Pytorch -> torch-mlir -> MLIR / IREE -> vulkan. Works on both Windows and Linux. And has a simple gradio web UI https://github.com/nod-ai/SHARK/tree/main/web https://github.com/nod-ai/SHARK/tree/main/web but we plan to enable better UI integrations very soon. Join us on discord https://discord.gg/RUqY2h2s9u https://discord.gg/RUqY2h2s9u if you have any trouble. Appreciate any / all feedback.

in3d 4y ago

In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it should be possible to fine tune this model to create them anyway using DreamBooth or a similar approach, they clearly went for the safe route after taking some heat. 1. https://twitter.com/emostaque/status/1595731407095140352?s=46&t=5Zxp66AzZrIrvccjqNMJPA https://twitter.com/emostaque/status/1595731407095140352?s=4...

poisonarena 4y ago
this is really disappointing
arez 4y ago
does this mean that stuff like artstation and deviantart doesn't work anymore as prompts? That would be a huge change
- in3d 4y ago
  Based on sample images I’ve seen, “Greg Rutkowski” doesn’t work anymore for example.
- siraben 4y ago
  Ran the model locally. Neither "trending on artstation" nor "Greg Rutowski" make any differences to the image anymore.[0] I suspect that people will find keywords that would improve the aesthetics further again, or that fine-tuning will also take place. [0] https://imgsli.com/MTM1ODQ5 https://imgsli.com/MTM1ODQ5
  - astrange 4y ago
    “Prompting” and “keywords” are not an essential part of this technology. If you like tokens, make your own tokens with textual-inversion or image inputs.
CuriouslyC 4y ago
Removing NSFW content is fine, people who care about that can work around it easily. Removing celebrities and commercial artists was a mistake though and I expect this will need to be really impressive in other ways or people aren't going to bother using it.

spcebar 4y ago

Well darn. This is an awesome leap, but I've spent the last few months making a card game using Stable Diffusion art and I guess now I need to go back and go over everything again. Congratulations to the SD team on another wonderful step forward!

mensetmanusman 4y ago

Highly recommend downloading draw anything on iOS, sd1.4 is available and it is quite fun!

devxpy 4y ago

Try it out here - https://app.gooey.ai/CompareText2Img/?example_id=1uONp1IBt0Y https://app.gooey.ai/CompareText2Img/?example_id=1uONp1IBt0Y

sssswww 4y ago

galactic freaks

sawirricardo 4y ago

Here are some which you'll encounter if you want to rabbit hole this trend - replicate.com - banana.dev - huggingface.co - lambdalabs.com - astriaAI - lexica.art for prompt inspiration