3 ms·

Great project, definitely a worthy goal. I know several people who are skeptical of voice assistants because they aren't fully on device. The blog article was

Great project, definitely a worthy goal. I know several people who are skeptical of voice assistants because they aren't fully on device. The blog article was very lean on the technical details apart from some basics about rust's safety and portability features. could you elaborate on where the gains from rust come into play ?, is your model built on a DL framework that is written in rust ?.

oulipo 9y ago
The DL framework is classical, but the models can then be reimplemented in pure Rust for efficiency
- quadrature 9y ago
  So you're taking the parameters from something like tensorflow and then exporting them to be executed in Rust ? and that is more efficient than the c++ backend of tensorflow ?.
  - throwawayendy 9y ago
    And also removing all of the overhead of the machine learning abstraction. Although I am skeptical that anyone would ever actually do that
    - quadrature 9y ago
      Not exactly sure what you mean. After the model is trained there isn't much "machine learning abstraction". you can serve the model using tensorflow serving using exported parameters. Its exactly what you want for serving the model.
    - danieldk 9y ago
      I used to train my parser (before switching to Tensorflow) using Caffe, dumped the parameters using a small program, and loaded them up in Go arrays slices and applied the network using simple C BLAS operations. This works fine, especially when you are using simpler networks. As a bonus, you don't have the overhead of Tensorflow session runs. It does become a bit of a drag when you are building more complex networks (e.g. with multiple RNN layers, batch normalization, etc.). In that case there are two straightforward options for Rust. There is a Tensorflow Rust binding against the tensorflow C API [1] with which you can just load and run a frozen Tensorflow graph. This is the approach that I am currently using, though I am running graphs on workstations/servers. Another option is compiling the graph with Tensorflow's XLA AOT compilation, which compiles the network to C++ classes (that you could bind from Rust). [1] https://github.com/tensorflow/rust https://github.com/tensorflow/rust
      - albi_lander 9y ago
        There seems to be a third option provided by someone, kali, working at Snips: https://github.com/kali/tensorflow-deploy-rust https://github.com/kali/tensorflow-deploy-rust
        danieldk 9y ago
        This only supports a small subset of ops. It is pretty much corresponds to the first option that I mentioned - train with Tensorflow, extract the parameters and provide implemenations of ops in your native language.
  - oulipo 9y ago
    It is useful for very small devices where you don't want to ship a bigger C++ lib
    - quadrature 9y ago
      Neat, would be great to have a technical blog post on what your team has built. it sounds really interesting.
      - jodureau 9y ago
        Hi! Keep an eye on our Medium: https://medium.com/snips-ai https://medium.com/snips-ai We have a few technical blog posts out there already, and a few more coming.
- tmzt 9y ago
  have you seen RLSL? It's a subset of Rust targetting SPIR-V. https://github.com/MaikKlein/rlsl https://github.com/MaikKlein/rlsl

oulipo 9y ago

The DL framework is classical, but the models can then be reimplemented in pure Rust for efficiency

quadrature 9y ago
So you're taking the parameters from something like tensorflow and then exporting them to be executed in Rust ? and that is more efficient than the c++ backend of tensorflow ?.
- throwawayendy 9y ago
  And also removing all of the overhead of the machine learning abstraction. Although I am skeptical that anyone would ever actually do that
  - quadrature 9y ago
    Not exactly sure what you mean. After the model is trained there isn't much "machine learning abstraction". you can serve the model using tensorflow serving using exported parameters. Its exactly what you want for serving the model.
  - danieldk 9y ago
    I used to train my parser (before switching to Tensorflow) using Caffe, dumped the parameters using a small program, and loaded them up in Go arrays slices and applied the network using simple C BLAS operations. This works fine, especially when you are using simpler networks. As a bonus, you don't have the overhead of Tensorflow session runs. It does become a bit of a drag when you are building more complex networks (e.g. with multiple RNN layers, batch normalization, etc.). In that case there are two straightforward options for Rust. There is a Tensorflow Rust binding against the tensorflow C API [1] with which you can just load and run a frozen Tensorflow graph. This is the approach that I am currently using, though I am running graphs on workstations/servers. Another option is compiling the graph with Tensorflow's XLA AOT compilation, which compiles the network to C++ classes (that you could bind from Rust). [1] https://github.com/tensorflow/rust https://github.com/tensorflow/rust
    - albi_lander 9y ago
      There seems to be a third option provided by someone, kali, working at Snips: https://github.com/kali/tensorflow-deploy-rust https://github.com/kali/tensorflow-deploy-rust
      - danieldk 9y ago
        This only supports a small subset of ops. It is pretty much corresponds to the first option that I mentioned - train with Tensorflow, extract the parameters and provide implemenations of ops in your native language.
- oulipo 9y ago
  It is useful for very small devices where you don't want to ship a bigger C++ lib
  - quadrature 9y ago
    Neat, would be great to have a technical blog post on what your team has built. it sounds really interesting.
    - jodureau 9y ago
      Hi! Keep an eye on our Medium: https://medium.com/snips-ai https://medium.com/snips-ai We have a few technical blog posts out there already, and a few more coming.
tmzt 9y ago
have you seen RLSL? It's a subset of Rust targetting SPIR-V. https://github.com/MaikKlein/rlsl https://github.com/MaikKlein/rlsl