2 ms·
It'd be great if it supports a wasm/web backend as well. I bet a lot of trivial text capabilities (grammar checking, autocomplete, etc) will benefit from this
by lewisjoe 10mo ago
It'd be great if it supports a wasm/web backend as well.
I bet a lot of trivial text capabilities (grammar checking, autocomplete, etc) will benefit from this rather than sending everything to a hosted model.
It's possible right now with onnx / transformers.js / tensorflow.js - but none of them are quite there yet in terms of efficiency. Given the target for microcontrollers, it'd be great to bring that efficiency to browsers as well.
- klaussilveira 10mo agoIf you need WASM, I think Candle is your current best bet: https://github.com/huggingface/candle https://github.com/huggingface/candle
- miguel_martin 10mo agoYou can compile to wasm, I have done so via the XNNPACK backend - you might have to tweak the compilation settings and upgrade the XNNPACK submodule/patch some code. But this only supports CPU, not a WebGPU or WebGL backend.