3 ms·
Why is only one of them on WebGPU? Is it because there additional tricky steps required to make a model work on WebGPU, or is there a limitation on what ops are
by jph00 3y ago
Why is only one of them on WebGPU? Is it because there additional tricky steps required to make a model work on WebGPU, or is there a limitation on what ops are supported there?
I'm keen to do more stuff with WebGPU, so very interested to learn about challenges and limitations here.
- xenova 3y agoWe have some other WebGPU demos, including: - WebGPU embedding benchmark: https://huggingface.co/spaces/Xenova/webgpu-embedding-benchmark https://huggingface.co/spaces/Xenova/webgpu-embedding-benchm... - Real-time object detection: https://huggingface.co/spaces/Xenova/webgpu-video-object-detection https://huggingface.co/spaces/Xenova/webgpu-video-object-det... - Real-time background removal: https://huggingface.co/spaces/Xenova/webgpu-video-background-removal https://huggingface.co/spaces/Xenova/webgpu-video-background... - WebGPU depth estimation: https://huggingface.co/spaces/Xenova/webgpu-depth-anything https://huggingface.co/spaces/Xenova/webgpu-depth-anything - Image background removal: https://huggingface.co/spaces/Xenova/remove-background-webgpu https://huggingface.co/spaces/Xenova/remove-background-webgp... You can follow the progress for full WebGPU support in the v3 development branch (https://github.com/xenova/transformers.js/pull/545 https://github.com/xenova/transformers.js/pull/545). To answer your question, while there are certain ops missing, the main limitation at the moment is for models with decoders... which are not very fast (yet) due to inefficient buffer reuse and many redundant copies between CPU and GPU. We're working closely with the ORT team to fix these issues though!