3 ms·
Not really a WebNN expert, but looks like it doesn't support CPU inference yet and only works in Chrome. It also lacks support for custom kernels, which we need
by om8 2y ago
Not really a WebNN expert, but looks like it doesn't support CPU inference yet and only works in Chrome. It also lacks support for custom kernels, which we need for running AQLM-quantized models.
When you ask about spec change, do you mean WebNN spec or something else?
- westurner 2y agoThere's WebNN, and WebGPU, but no WebTPU; so I guess WebNN would be it "Web Neural Network API" W3C Candidate Recommendation Draft https://www.w3.org/TR/webnn/ https://www.w3.org/TR/webnn/ "WebGPU" W3C Candidate Recommendation Snapshot: https://www.w3.org/TR/webgpu/ https://www.w3.org/TR/webgpu/ WebGPU: https://en.wikipedia.org/wiki/WebGPU https://en.wikipedia.org/wiki/WebGPU Tested koboldcpp with a 7B GGUF model because it should work with 8Gb VRAM; but FWIU somehow kobold pages between RAM and GPU VRAM. How does the user, in WASM, know that they have insufficient RAM for a model?