12 ms·
Run Stable Diffusion on Intel CPUs
- Aiedail 4y agoI'm curious about what makes this project special, see there's a lot of similar implementations of diffusion models based on pytorch/tf. Is it because it use the cpu itself to produce the diffusion process?
- nperez 4y agoYeah. For something like this, you ideally would want a powerful GPU with 12-24gb VRAM. If you have something like an RTX 2070 at the bare minimum, you probably don't need this and could do a lot more steps a lot faster on a GPU, but it's great for those who don't have that option.
- Scaevolus 4y agoA $500 RTX 3070 with 8GB of VRAM can generate 512x512 images with 50 steps in 7 seconds.
- ShamelessC 4y agoThe RTX 2070 also shipped with 8GB of VRAM, just fyi.
- nperez 4y agoYep, 8GB works fine. The 2070 is where I started. I wouldn't consider it ideal, though. There will be cases where you'll wish you could increase the resolution a little more, or could do just a few more per batch, but you're getting CUDA out-of-memory errors
- motoboi 4y agoopenvino is an unsung hero.
- amrrs 4y agoOn reddit I found some older GPUs take about 5 mins and here this video[1] says 5 mins for CPU using this OpenVino library. Not sure if OpenVino makes CPU chips compete with GPUs. Has anyone heard of OpenVino before ? 1.https://youtu.be/5iXhhf7ILME https://youtu.be/5iXhhf7ILME
- T-A 4y agohttps://github.com/openvinotoolkit/openvino#supported-hardware-matrix https://github.com/openvinotoolkit/openvino#supported-hardwa...
- minimaxir 4y agoOpenVINO is developed by Intel themselves, and is one of many methods to freeze models to make CPU inference possible and performant. https://en.wikipedia.org/wiki/OpenVINO https://en.wikipedia.org/wiki/OpenVINO
- yayr 4y agothen - how far away are we from having it on M1/M2 Macs, at least with regular processing? openvino may be one path I suppose: https://github.com/openvinotoolkit/openvino/issues/11554 https://github.com/openvinotoolkit/openvino/issues/11554
- yayr 4y agolooks like there is an easier path using metal shaders: https://dev.to/craigmorten/setting-up-stable-diffusion-for-macos-1igl https://dev.to/craigmorten/setting-up-stable-diffusion-for-m... and https://github.com/magnusviri/stable-diffusion/tree/apple-silicon-mps-support https://github.com/magnusviri/stable-diffusion/tree/apple-si...
- garblegarble 4y agoI've been using this on my M1 Max and it works pretty well, 1.65 iterations per second (full precision, whereas my PC's 3080 can only do half-precision due to limited memory)... a 50-iteration image in about 40 seconds or so.
- smoldesu 4y ago> full precision, whereas my PC's 3080 can only do half-precision due to limited memory What model are you using? I've been running full-precision SD1.4 on my 3070, albeit with less than 10% VRAM headroom.
- MattRix 4y agoYour 3080 should be able to do full precision. Are you sure you don’t have the batch size set greater than 1, or another issue along those lines?
- garblegarble 4y agoThank you and smoldesu for letting me know it should work, I'll have a better look into what's going on - it didn't immediately work on Windows in full precision (probably a batch size issue as you suggested) and I gave up... I shouldn't have given up so easily, but my tolerance for annoyances on Windows is pretty low (that Windows machine is kept for gaming, the last time I used a Windows machine for anything but launching Steam was when Windows 2000 was the hot new thing...)
- ByThyGrace 4y agoWhat's the status of running SD on AMD GPUs?
- homarp 4y agohttps://rentry.org/tqizb https://rentry.org/tqizb explains how to install ROCm and then pytorch for ROCm ROCm does not support APU, here is the list of supported GPU: https://docs.amd.com/bundle/Hardware_and_Software_Reference_Guide/page/Hardware_and_Software_Support.html https://docs.amd.com/bundle/Hardware_and_Software_Reference_...
- synergy20 4y agowhat does APU mean here?
- ace2358 4y agoCpu + gpu on the same die/chiplet thing. So integrated gpu in and marketing speak is apu
- barkingcat 4y agoAmd's integrated gpu together with the processor.
- panki27 4y agoTried it, could not get it to run on my RX570. Read conflicting information whether that series is still supported or not. rocm-smi detects the card and shows me the temps etc., but rocminfo throws an error. I found this repo which should apparently fix it, didn't change my situation though: https://github.com/xuhuisheng/rocm-gfx803/ https://github.com/xuhuisheng/rocm-gfx803/
- yieldcrv 4y agoWhere can i get up to speed on what’s coming down there pipeline in this ai/ml image making scene? (And learn the agreed upon terms)
- aaaaaaaaaaab 4y agoNoone can tell. Pandora's box has been opened. Nothing is true, everything is permitted.
- desindol 4y agoIt’s mostly all on discord. These are the two most active ones with devs on board https://discord.gg/QNxzjUfu https://discord.gg/QNxzjUfu https://discord.gg/nZ3hkXRV https://discord.gg/nZ3hkXRV
- adultSwim 4y agoTwitter too
- yieldcrv 4y agoany profiles I should follow? just need a couple so that the algorithm suggests the other ones and the rabbit hole forms itself
- adultSwim 4y agoI suggest starting with the companies and key public employees. https://twitter.com/EMostaque https://twitter.com/EMostaque https://twitter.com/RiversHaveWings https://twitter.com/RiversHaveWings https://twitter.com/laion_ai https://twitter.com/laion_ai https://twitter.com/StableDiffusion https://twitter.com/StableDiffusion https://twitter.com/openart_ai https://twitter.com/openart_ai Hashtags: #stablediffusion #dalle #aiart #generativeart For a taste of what's coming further down the pike, I recommend senior ML researchers. https://twitter.com/boredbengio https://twitter.com/boredbengio https://twitter.com/jure https://twitter.com/jure
- 4y ago
- polskibus 4y agocan't get it to install requirements on Windows with Python 3.10 and MS Build Tools 2022. Any tips?
- smoldesu 4y agoI found a pretty good Docker container for it, though that's only really switching you from solving Python problems to Docker ones. Worth trying out if you have a Linux box or WSL installed though: https://github.com/AbdBarho/stable-diffusion-webui-docker https://github.com/AbdBarho/stable-diffusion-webui-docker
- desindol 4y agoIt needs python 3.9.
- manyone1 4y agopython 3.10 will fail on openvino. i used these steps: anaconda prompt cd to the destination folder conda create --name py38 python=3.8 conda activate py38 conda update --all conda install openvino-ie4py -c intel pip install -r requirements.txt i also had to edit stable_diffusion.py and changed in the #decoder area: changed vae.xml and vae.bin to vae_decoder.xml and vae_decoder.bin respectively from there i could run python stable_diffusion.py --prompt "Street-art painting of Emma Stone dancing, in picasso style" for img2img, use this (note DIFFERENT program): python demo.py --prompt "astronaut with jetpack floating in space with earth below" --init-image ./data/jomar.jpg --strength 0.5
- aaaaaaaaaaab 4y agoLove this. OpenAI are livid. :^)
- enchiridion 4y agoWhy?
- aaaaaaaaaaab 4y agoBecause they no longer control the narrative.
- Gigachad 4y agoIt does ruin the illusion that you need crazy million dollar servers to run the model and the world would fall in to chaos if the public got their hands on these models.
- optimalsolver 4y agoTo be fair, the world only just got their hands on this (and by world I mean people with decent hardware), so too soon to say what the ramifications will be.
- bogwog 4y agoAlso to be fair, the job "AI ethicist" probably didn't exist as a real thing until a few years ago. So the people in those roles over at OpenAI likely have no idea what they're doing.
- adultSwim 4y agoWe wouldn't be able to run it ourselves if they hadn't trained it on 4000 GPUs for a month.
- xtreme 4y ago
- torotonnato 4y ago7' 12" on an ancient Intel Core i5-3350P CPU @ 3.10GHz (!) using BERT BasicTokenizer, default arguments
- deleted 4y ago[deleted]
- mysterydip 4y agoI didn't see any requirements on the page beyond a CPU on that list. Do you need a certain amount of RAM? Will more speed things up to a degree?
- neurostimulant 4y agoIt used ~8GB of ram on my machine with similar generation time to the low vram fork of stable diffusion [1] running on my 4GB GTX1650. [1] https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion
- keepquestioning 4y agoAnything for M1 GPU?
- hustwindmaple1 4y agoIt works just fine.
- avocado2 4y agoweb demo for stable diffusion (txt2img): https://huggingface.co/spaces/stabilityai/stable-diffusion https://huggingface.co/spaces/stabilityai/stable-diffusion github with web ui: https://github.com/hlky/stable-diffusion https://github.com/hlky/stable-diffusion dev repo (more features, may have bugs): https://github.com/hlky/stable-diffusion-webui https://github.com/hlky/stable-diffusion-webui repo with docker: https://github.com/AbdBarho/stable-diffusion-webui-docker https://github.com/AbdBarho/stable-diffusion-webui-docker colab repo (new): https://github.com/altryne/sd-webui-colab https://github.com/altryne/sd-webui-colab can also run it in colab (includes img2img): https://colab.research.google.com/drive/1NfgqublyT_MWtR5Csmr https://colab.research.google.com/drive/1NfgqublyT_MWtR5Csmr demo made with gradio: https://github.com/gradio-app/gradio https://github.com/gradio-app/gradio
- skybrian 4y agoOr if you don’t want to tweak anything, just use the hosted version? https://beta.dreamstudio.ai/ https://beta.dreamstudio.ai/
- esperent 4y agoIt's pretty expensive. They give you 100 free credits but I burned though that in about 10 minutes just trying to figure out how things worked. Didn't get any nice images. After that, it's $1 per 100 credits, so about $6 an hour maybe.
- skybrian 4y agoI put $10 in and it’s lasted a long time, but I don’t generate images every day. Cheaper than getting a good graphics card, anyway.
- ConceptJunkie 4y agoOne credit is one regular-sized picture generation. If you want to make it bigger or run more steps (in my experience, this is less useful than it would seem), then it can run up to 3 or more credits per generation. That's still 3 cents a picture. Compared to Dall-E and even Midjourney, that's crazy cheap.
- ycombinatorrio 4y agoFor those who wants to know before installing on my 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz, it consumes 13 GB of RAM and takes 4 minutes to generate an image with 32 steps (~2 min for 16 steps).
- yyyk 4y agoOn Linux/i5-1135G7 - takes 3min very consistently for 32 steps. Memory use: ~13.5gb VIRT, 9.3gb RES.
- digitallyfree 4y agoOn my Intel i5-4590T (8G RAM) it takes around 5-6 minutes to generate with 32 steps, swapping to disk as it does consume around 13G memory total. You don't get real-time feedback but it's very usable and fun to play with. I wish there was an option to force a manual seed though.
- digitallyfree 4y agoFYI they just added the seed option to the repo today. Now waiting for img2img...
- stateoff 4y agohttps://laion.ai/faq/ https://laion.ai/faq/ Based on the FAQ of the dataset that was used for training of https://huggingface.co/spaces/stabilityai/stable-diffusion https://huggingface.co/spaces/stabilityai/stable-diffusion LAION datasets are simply indexes to the internet, i.e. lists of URLs to the original images together with the ALT texts found linked to those images. While we downloaded and calculated CLIP embeddings of the pictures to compute similarity scores between pictures and texts, we subsequently discarded all the photos. Any researcher using the datasets must reconstruct the images data by downloading the subset they are interested in. For this purpose, we suggest the img2dataset tool. I love the "*simply*", but doesn't it mean that (depending on country, laws etc., but generally): 1. The LAION group committed possible copyright infringements and even left undeniable evidence that they did - on top of their written testimony (dumping the "stolen goods into the river" does not make the infringement undone, does it?) 2. Any model trained on the "linked" data may commit copyright infringement. 3. As consequence, you using generated images may be liable. I always wonder how it possibly is legal at all - considering that as a human artist if I was to copy material and remix it it without proper permission would be liable (again depending on situation), but suddenly ML is around the corner and it's all great and now you can keep remixing the potential problematic output further - no questions asked!? I guess there are no precedence cases but why should an automaton/software (and its creators) be judged differently to persons? I don't want to spoil the fun but what am I missing? Also disappointed that this dataset did not make sure to only collect unproblematic content like Creative Commons that allows remixing. Would be a hell of a attribution list but definitely better than what is presented here. EDIT: Formatting EDIT2: I actually followed one of the projects mentioned not the linked repository. Clarified above.
- pdntspa 4y agoIs feeding copyrighted material into an AI really copyright infringement?
- moosedev 4y agoI don't know, but it's quite entertaining when the output occasionally has a corrupted, but recognizable, Getty Images watermark: https://imgur.com/SmibVME https://imgur.com/SmibVME (Prompt: "A horse delivering mail in New York City, 1870")
- hustwindmaple1 4y agoIt is noticeably faster than the original model (~30-40%) on my machine.
- moosedev 4y agoWorking in WSL (Windows 10) Ubuntu on Ryzen 5600X; uses ~11GB of RAM and takes 2m04s with the default settings. This is the first time I've played with a text-to-image model. I was aware that so-called "prompt engineering" can be tricky, but it's wild to see it for myself. A single space character can be the difference between garbage (or nightmare-fuel) output and output that captures the spirit of the prompt pretty well.
- teruakohatu 4y ago> A single space character can be the difference between garbage (or nightmare-fuel) output and output that captures the spirit of the prompt pretty well. It shouldn't really, have you tried generating a few images with each prompt with and without space? Even with the same prompt, you can get a wide variety of quality.
- nl 4y agoThat's very surprising and shouldn't be the case in general (the exception being things like compound words or spelling errors maybe). Do you have some examples? Are you fixing the random seed? If not the variation is more likely to be that than a single space.
- LanternLight83 4y ago> Ryzen 5600X Ooh, I've got one of those! I've been getting by trying to run it on my PC, which for various reasons currently has a 5800 and a GTX 1050 4GB, which can just bearly handle optimizedSD at 90s/image but runs out of memory if I try to use the popular webui repo. Swapping to the 5600X might be worth it!
- boppo1 4y agoThe most powerful device I have is an ipad pro M1 16gb ram. Can I run this on that thing at all?
- krasi0 4y agoGreat work! Is there a similar project for (local) text generation (NLP) on a CPU + lots of RAM. I mean something transformers-based and of similar quality to GPT-3 (i.e. better than GPT-2). I understand that each prompt would take almost forever to complete but still curious if something like that exists
- versteegen 4y agoYes. Fabrice Bellard wrote a highly optimised library (libnc) [1] for training and inference of neural networks on CPU (x86 with AVX-2), and implemented GPT-2 inference (gpt2tc) with it [2]. Later he added a CUDA backend to libnc. You can try it out at his website TextSynth [3] and I see it now runs various newer GPT-based models too, but it seems he hasn't released the code for that. Doesn't surprise me as he didn't release the code for libnc either, just the parts of gpt2tc excluding libnc (libnc is released as a free binary) so someone could reimplement GPT-J and the other models themselves. Incidentally, he's currently leading the Large Text Compression Benchmark using a -based compressor called nncp [4] which is based on this work. It learns the transformer-based model as it goes, and the earlier versions didn't use a GPU. [1] https://bellard.org/libnc/ https://bellard.org/libnc/ [2] https://bellard.org/libnc/gpt2tc.html https://bellard.org/libnc/gpt2tc.html [3] https://textsynth.com/ https://textsynth.com/ [4] http://www.mattmahoney.net/dc/text.html#1085 http://www.mattmahoney.net/dc/text.html#1085
- krasi0 4y agoYet another gem by the genius Fabrice B! I kinda understand why he would not release the source code. Perhaps, he's finally decided to monetize some of his coding skills. Maybe in the future, he'll start releasing some of those newer and bigger models to the public given that other big corps like FB have started already doing so (GPT-NeoX and OPT - as mentioned in the sibling comment by infinityio)
- versteegen 4y agoYes, TextSynth.com is a commercial service, see pricing [1]. If his code is faster than others' (I'd certainly believe it) then it's quite valuable, and he deserves to be able to monetise it. Edit: Also, OpenAI is slashing price for GPT-3 by 2-3x tomorrow because of "progress in making our models more efficient to run" [2]. Also, he was/is competing for the Hutter Prize with nncp, however he is outside the requirements for the prize: CPU-time, RAM, but most especially that submissions shouldn't require a modern CPU (with AVX-2) or a GPU. Otherwise he could have won it. I suspect it's actually that's the biggest reason he implemented libnc without GPU support initially. He has asked for the rules to be changed to allow AVX2 and I believe they eventually will be. So he won't give away the source for nncp yet, but will have to open source it to receive the prize. [1] https://textsynth.com/pricing.html https://textsynth.com/pricing.html [2] https://help.openai.com/en/articles/6485334-openai-api-pricing-update-faq https://help.openai.com/en/articles/6485334-openai-api-prici...
- ManuelKiessling 4y agoI've set up a Discord Bot that turns your text prompt into images using Stable Diffusion. You can invite the bot to your server via https://discord.com/api/oauth2/authorize?client_id=1013373043625705513&permissions=117760&scope=bot%20applications.commands https://discord.com/api/oauth2/authorize?client_id=101337304... Talk to it using the /draw Slash Command. It's very much a quick weekend hack, so no guarantees whatsoever. Not sure how long I can afford the AWS g4dn instance, so get it while it's hot. PS: Anyone knows where to host reliable NVIDIA-equipped VMs at a reasonable price?
- ManuelKiessling 4y agoOh and get your prompt ideas from https://lexica.art https://lexica.art if you want good results.
- deleted 4y ago[deleted]
- rhn_mk1 4y agoWhat OS does this need? Using Ubuntu 20.04, I'm getting stuck on openvino: > $ pip install > Could not find a version that satisfies the requirement openvino==2022.1.0 (from -r requirements.txt (line 6)) (from versions: 2021.4.0, 2021.4.1, 2021.4.2) I even upgraded to python3.9, which, inexplicably, is required but not available in the "supported" OS. EDIT: apparently it requires a version of pip that's newer than the one bundled with Ubuntu.