I used to train my parser (before switching to Tensorflow) using Caffe, dumped the parameters using a small program, and loaded them up in Go arrays slices and

I used to train my parser (before switching to Tensorflow) using Caffe, dumped the parameters using a small program, and loaded them up in Go arrays slices and applied the network using simple C BLAS operations. This works fine, especially when you are using simpler networks. As a bonus, you don't have the overhead of Tensorflow session runs. It does become a bit of a drag when you are building more complex networks (e.g. with multiple RNN layers, batch normalization, etc.). In that case there are two straightforward options for Rust. There is a Tensorflow Rust binding against the tensorflow C API [1] with which you can just load and run a frozen Tensorflow graph. This is the approach that I am currently using, though I am running graphs on workstations/servers. Another option is compiling the graph with Tensorflow's XLA AOT compilation, which compiles the network to C++ classes (that you could bind from Rust). [1] https://github.com/tensorflow/rust https://github.com/tensorflow/rust