3 ms·
These are small enough that you can run them entirely in the browser https://huggingface.co/spaces/webml-community/ternary-bonsai-2-webgpu-kernels https://huggi
by Aurornis 18d ago
These are small enough that you can run them entirely in the browser https://huggingface.co/spaces/webml-community/ternary-bonsai-2-webgpu-kernels https://huggingface.co/spaces/webml-community/ternary-bonsai...
Remember to clear the downloaded weights afterward.
Like the last model, it's amazing they work as well as they do. Use it for any longer task and they fall apart spectacularly and in interesting ways.
- outofpaper 18d agoSo you have some fun examples?
- Aurornis 17d agoMaybe I oversold the fun-ness of it. The most common failure mode is that it goes into loops and you come back to find it exhausted the output length without getting anywhere.
- SXX 18d agoSadly crashing on Pixel 9 Pro, but I guess phone GPU with 16GB RAM total wouldnt be enough anyway.
- trvz 18d agoIt should be though.
- SXX 18d agoEven native AI gallery uses smaller Gemma models. I guess whatever Chrome is using for WebGPU compute on Android is just adding too much overhead.
- 14u2c 18d agoRuns on my 16GB M2 Air (firefox). ~7 tok/s
- jeroenhd 17d ago3/16 GiB of RAM on the P9Pro is dedicated towards on-device AI models, which WebGPU probably can't access.
- throwa356262 17d agoTo be fair, the GPU in pixel 9 and 10 is pretty bad. I really hope they stop using PowerVR in pixel 11.
- aschobel 17d ago40 tokens a second on my M4 Max (Safari). Wild times.