3 ms·

The serialisation story in Tensorflow is an obscene mess. There are bugs open on keras and tensorflow asking how to export a model and run it on your laptop and

The serialisation story in Tensorflow is an obscene mess. There are bugs open on keras and tensorflow asking how to export a model and run it on your laptop and even better...on Android. It simply is crazy bad and cannot be done easily. In fact, to do even half decent export of TF models, you have to switch to keras to try and do any kind of export. I have a 10 email conversation with enterprise Google Cloud support to try and get a ML Engine output serialised to work on Android. There are threads open all over the place on stackoverflow and elsewhere - and yes, we have tried all SIX ways.

danieldk 9y ago
Out of curiosity, what problems are you running into? I have never had serious problems with the 'save parameters -> dump graph -> freeze graph -> load up with C API' path with feed-forward networks or various RNNs. Either from Go or from Rust. Admittedly, the documentation in this area is extremely bad and I basically had to figure out myself how to do it, though this was long before 1.0.
- sandGorgon 9y ago
  The issues with the high level API goes slightly deeper. It looks like some of the graph operations are not available on Android (and equivalently on the desktop) by default[1]. The motivation for this is that we have stricter requirements for computation costs and application size on different platforms. So there is an approach that allows to compile minimal set of operations required to run the graph[2][3]. However it requires Bazel as the primary build tool for the Android app as well. The good news is that the TensorFlow team understands the issue and works on improving the documentation and tooling[4]. That said, would you be able to share any example snippets on how you are persisting and loading these models in your code ? That would be super helpful. Also, im getting the feeling that you are using the deprecated method of saving. I think they are shifting to Metagraph now (not sure about this) https://www.tensorflow.org/versions/master/api_docs/python/tf/train/export_meta_graph https://www.tensorflow.org/versions/master/api_docs/python/t... [1] https://github.com/tensorflow/tensorflow/issues/10254 https://github.com/tensorflow/tensorflow/issues/10254 [2] https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/print_selective_registration_header.py https://github.com/tensorflow/tensorflow/blob/master/tensorf... [3] https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/framework/selective_registration.h https://github.com/tensorflow/tensorflow/blob/master/tensorf... [4] https://github.com/tensorflow/tensorflow/issues/10299 https://github.com/tensorflow/tensorflow/issues/10299
  - danieldk 9y ago
    That said, would you be able to share any example snippets on how you are persisting and loading these models in your code ? The relevant code is (still) in private repositories, but I have a deck with some examples using Rust: https://www.dropbox.com/s/t8r056f6wqlktqv/embedding-tensorflow.pdf?dl=0 https://www.dropbox.com/s/t8r056f6wqlktqv/embedding-tensorfl...

danieldk 9y ago

Out of curiosity, what problems are you running into? I have never had serious problems with the 'save parameters -> dump graph -> freeze graph -> load up with C API' path with feed-forward networks or various RNNs. Either from Go or from Rust. Admittedly, the documentation in this area is extremely bad and I basically had to figure out myself how to do it, though this was long before 1.0.

sandGorgon 9y ago
The issues with the high level API goes slightly deeper. It looks like some of the graph operations are not available on Android (and equivalently on the desktop) by default[1]. The motivation for this is that we have stricter requirements for computation costs and application size on different platforms. So there is an approach that allows to compile minimal set of operations required to run the graph[2][3]. However it requires Bazel as the primary build tool for the Android app as well. The good news is that the TensorFlow team understands the issue and works on improving the documentation and tooling[4]. That said, would you be able to share any example snippets on how you are persisting and loading these models in your code ? That would be super helpful. Also, im getting the feeling that you are using the deprecated method of saving. I think they are shifting to Metagraph now (not sure about this) https://www.tensorflow.org/versions/master/api_docs/python/tf/train/export_meta_graph https://www.tensorflow.org/versions/master/api_docs/python/t... [1] https://github.com/tensorflow/tensorflow/issues/10254 https://github.com/tensorflow/tensorflow/issues/10254 [2] https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/print_selective_registration_header.py https://github.com/tensorflow/tensorflow/blob/master/tensorf... [3] https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/framework/selective_registration.h https://github.com/tensorflow/tensorflow/blob/master/tensorf... [4] https://github.com/tensorflow/tensorflow/issues/10299 https://github.com/tensorflow/tensorflow/issues/10299
- danieldk 9y ago
  That said, would you be able to share any example snippets on how you are persisting and loading these models in your code ? The relevant code is (still) in private repositories, but I have a deck with some examples using Rust: https://www.dropbox.com/s/t8r056f6wqlktqv/embedding-tensorflow.pdf?dl=0 https://www.dropbox.com/s/t8r056f6wqlktqv/embedding-tensorfl...