4 ms·

Running Stable Diffusion fully in browser with WebGPU

42 points by crowwork 4y ago

crowwork 4y ago
This project brings stable diffusion models to web browsers. Everything runs inside the browser with no server support. Please check out our GitHub repo to see how we did it. There is also a demo which you can try out.
jinhongyii 4y ago
Great work! It's exciting to see such a large model running on my local web browser.
sberens 4y ago
> As WebGPU does not support FP16 at this moment, the memory consumption of running the demo is about 7GB. For Apple silicon Mac with only 8GB of unified memory, it may take longer (a few minutes) to generate an image. This demo may also work for Mac with AMD GPU. Ouch. It will be interesting what this number will look like in 2-3 years though.
- crowwork 4y ago
  on apple M2max, it takes 20sec, and with the fp16 support, there will be opportunities for improvement likely soon this eyar
sberens 4y ago
I'm looking forward to an OpenAI quality embedding that I can use locally. I think eliminating the latency/cost from calling their API will allow for some novel UX patterns.
- crowwork 4y ago
  The current demo uses CLIP model from openai, which is likely what you are looking for
  - sberens 4y ago
    Oops I meant text embeddings
spectrometerHBH 4y ago
50s per image, 16'' M1 pro 16G mbp, amazing!
cyx6 4y ago
This is an AMAZING work! Just type in one line description and click, I get a photo by stable diffusion on my macbook in minutes. And it never needs any setup, just having fun. :)
l3jin 4y ago
Cooool! We can use Stable Diffusion models on chrome. It can generate any image I want within 2mins:)
junrushao1994 4y ago
This actually works on my non-M1/M2 macbook with AMD GPUs pretty smoothly. It's a bit slow for now (~70sec), but I expect this to be faster with fp16. Any roadmap to support fp16 on WebGPUs?
- crowwork 4y ago
  FP16 is already in the spec and hopefully will ship this year we believe
ziheng 4y ago
Cool project. The main point I guess is this doesn't require user to do any environment setup. All your need to do is just run it in the browser. One thing I am curious, how's the performance of WebGPU approach compared to the local server approach?
- crowwork 4y ago
  On the current setup(M2 and metal), it should run as fast as the local native environment.
zxybazh 4y ago
Very cool demo, got it working on M1 macbook, generates each pic in 140s. Fun experience without specific setup. Does it also work for cuda device? Is there any plan to support other models soon?
misakamisuzu 4y ago
Really impressive! Very smooth experience on chrome.
yongwww 4y ago
Great project! It is able to run on my local Mac M2, the generated result looks good to me! Looking forward to seeing more models are enabled
wuweil 4y ago
super cool! it also works on my MacBook with AMD GPU. this is a great demonstration of the power of web GPU
yecomb 4y ago
huggingface would save a lot of money if they know this :)

42 points by crowwork 4y ago

This project brings stable diffusion models to web browsers. Everything runs inside the browser with no server support. Please check out our GitHub repo to see how we did it. There is also a demo which you can try out.

jinhongyii 4y ago

Great work! It's exciting to see such a large model running on my local web browser.

sberens 4y ago

> As WebGPU does not support FP16 at this moment, the memory consumption of running the demo is about 7GB. For Apple silicon Mac with only 8GB of unified memory, it may take longer (a few minutes) to generate an image. This demo may also work for Mac with AMD GPU. Ouch. It will be interesting what this number will look like in 2-3 years though.

crowwork 4y ago
on apple M2max, it takes 20sec, and with the fp16 support, there will be opportunities for improvement likely soon this eyar

sberens 4y ago

I'm looking forward to an OpenAI quality embedding that I can use locally. I think eliminating the latency/cost from calling their API will allow for some novel UX patterns.

crowwork 4y ago
The current demo uses CLIP model from openai, which is likely what you are looking for
- sberens 4y ago
  Oops I meant text embeddings

spectrometerHBH 4y ago

50s per image, 16'' M1 pro 16G mbp, amazing!

cyx6 4y ago

This is an AMAZING work! Just type in one line description and click, I get a photo by stable diffusion on my macbook in minutes. And it never needs any setup, just having fun. :)

l3jin 4y ago

Cooool! We can use Stable Diffusion models on chrome. It can generate any image I want within 2mins:)

junrushao1994 4y ago

This actually works on my non-M1/M2 macbook with AMD GPUs pretty smoothly. It's a bit slow for now (~70sec), but I expect this to be faster with fp16. Any roadmap to support fp16 on WebGPUs?

crowwork 4y ago
FP16 is already in the spec and hopefully will ship this year we believe

ziheng 4y ago

Cool project. The main point I guess is this doesn't require user to do any environment setup. All your need to do is just run it in the browser. One thing I am curious, how's the performance of WebGPU approach compared to the local server approach?

crowwork 4y ago
On the current setup(M2 and metal), it should run as fast as the local native environment.

zxybazh 4y ago

Very cool demo, got it working on M1 macbook, generates each pic in 140s. Fun experience without specific setup. Does it also work for cuda device? Is there any plan to support other models soon?

misakamisuzu 4y ago

Really impressive! Very smooth experience on chrome.

yongwww 4y ago

Great project! It is able to run on my local Mac M2, the generated result looks good to me! Looking forward to seeing more models are enabled

wuweil 4y ago

super cool! it also works on my MacBook with AMD GPU. this is a great demonstration of the power of web GPU

yecomb 4y ago

huggingface would save a lot of money if they know this :)