4 ms·
There's also tract, from Sonos[0]. 100% Rust[1]. I'm currently using it to do speech recognition with a variant of the Conformer architecture (exported to ONNX
by davidatbu 4y ago
There's also tract, from Sonos[0]. 100% Rust[1].
I'm currently using it to do speech recognition with a variant of the Conformer architecture (exported to ONNX).
The final goal is to do it in WASM client-side.
[0] https://github.com/sonos/tract https://github.com/sonos/tract
[1] To be clear, I'm not saying that writing in Rust is some inherent advantage for an NN inference engine. But I personally gravitate to using libraries written in languages I know/want to learn, and Rust might be that for some people.
- brrrrrm 4y agoFor inference, models should be compiled down directly to WASM or WebGPU or whatever, right? The driving language really shouldn't matter at the end of the day. Unless you've got massive compute bound transformers or old-school full convolutions, if you're interpreting a list of operations you're going to lose perf.
- davidatbu 4y ago> For inference, models should be compiled down directly to WASM or WebGPU I mean, yes? Whether for inference or for training, ML operators are usually implemented in compiled low level languages. That implementation might also take advantage of GPUs. > ... if you're going interpreting a list of operations ... I'm not sure what "interpreting a list of operators" means here. To clarify why I mentioned the implementation language, I recently had to modify the source code of Tract to get my model running (I will push that change upstream at some point). I would not have (been able to do that), had tract been written in C++, for example.