5 ms·

Been using Tensorflow embedded in a mobile app for a few months and honestly, I’m constantly surprised at how well thought-out the tooling is, and how quickly y

Been using Tensorflow embedded in a mobile app for a few months and honestly, I’m constantly surprised at how well thought-out the tooling is, and how quickly you can get results. Conversely, I think a few things are still unnecessarily dense (installing dependencies, optimizing hyper-parameters, and some of the embedded/XLA stuff is very raw). Kudos to the team though. It sounds like they’re on the right track with TF overall, and focusing on performance (including the XLA stuff) + ease of use (high-level, Keras API) is absolutely what I want as a user right now. Keep up the great work, y’all.

syntaxing 10y ago
Would you happen to know if it requires additional code to support the Hexagon digital signal processor (DS)from Qualcomm Or is it automatic (kinda like switching between Tensorflow-CPU and Tensorflow-GPU)? I mainly work with Tensorflow on a PC so I'm not too familiar with the embedded variants of Tensorflow. Thanks!
- timanglade 10y ago
  I don’t have any experience with that unfortunately. I’ve seen a couple of talks/demos/announcements about it and it sounds like it’s automatic, but I haven’t been able to find the SDK or any tutorial for it, so I’m not 100% sure. The Qualcomm speaker this morning said there would be more details about it later today but I don’t see anything on the Agenda [0]. Maybe Pete’s session at 12:40 will cover it? [0]: https://events.withgoogle.com/tensorflow-dev-summit/agenda/#content https://events.withgoogle.com/tensorflow-dev-summit/agenda/#...
  - petewarden 10y ago
    I'm not going to cover it in detail in my talk, but the code with some barebones documentation is available at https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/hvx https://github.com/tensorflow/tensorflow/tree/master/tensorf... One thing to note is that this isn't available on production phones yet, because we need a signed driver to run within Android. You should be able to run this on a Dragonboard 820 development board though, using the instructions in the README. This is all very new though, so apologies in advance for any hiccups getting up and running. My email's petewarden at google.com if you are trying this and hit problems.
    - timanglade 10y ago
      Thanks Pete :)
    - syntaxing 10y ago
      Thanks Tim and Pete!
codesternews 10y ago
which app may i know?
- timanglade 10y ago
  Not released yet, should go live in April/May!
redsummer 10y ago
How do you embed in a mobile app?
- timanglade 10y ago
  Essentially there are two ways to do this. The “old” way is to export your TensorFlow neural network into a protobuf file, then load up the TensorFlow interpreter in your iOS/Android app, feed it the neural net, and run the inference directly on device. The GitHub repo [0] has a good set of examples of what that looks like in practice. The new, still experimental way is to compile your neural net into executable code with their XLA / tfcompile tool, and link that into your app. They are adding more docs on this on the TensorFlow website [1]. [0]: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/ios_examples https://github.com/tensorflow/tensorflow/tree/master/tensorf... [1]: https://www.tensorflow.org/versions/master/experimental/xla/tfcompile https://www.tensorflow.org/versions/master/experimental/xla/...
  - nojvek 10y ago
    Running neural nets on a standard mobile device will be game changing. I can't wait for mobile devices having custom chips to do AI related tasks. I'm predicting in a decade we'll have offline speech and image recognition running on the phone.
    - xtf 10y ago
      They don't want you to go there. Remind who started the latest big AI Projects (Google, Amazon, Microsoft, Facebook) and I don't think the will stop grabbing data. I think they'll develop a hivemind, where mobile adds to the pool. In short Skynet ;)
      - nojvek 10y ago
        Hopefully the ASICs are fast and open enough so we can see some open AIs who respect our privacy and security concerns

syntaxing 10y ago

Would you happen to know if it requires additional code to support the Hexagon digital signal processor (DS)from Qualcomm Or is it automatic (kinda like switching between Tensorflow-CPU and Tensorflow-GPU)? I mainly work with Tensorflow on a PC so I'm not too familiar with the embedded variants of Tensorflow. Thanks!

timanglade 10y ago
I don’t have any experience with that unfortunately. I’ve seen a couple of talks/demos/announcements about it and it sounds like it’s automatic, but I haven’t been able to find the SDK or any tutorial for it, so I’m not 100% sure. The Qualcomm speaker this morning said there would be more details about it later today but I don’t see anything on the Agenda [0]. Maybe Pete’s session at 12:40 will cover it? [0]: https://events.withgoogle.com/tensorflow-dev-summit/agenda/#content https://events.withgoogle.com/tensorflow-dev-summit/agenda/#...
- petewarden 10y ago
  I'm not going to cover it in detail in my talk, but the code with some barebones documentation is available at https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/hvx https://github.com/tensorflow/tensorflow/tree/master/tensorf... One thing to note is that this isn't available on production phones yet, because we need a signed driver to run within Android. You should be able to run this on a Dragonboard 820 development board though, using the instructions in the README. This is all very new though, so apologies in advance for any hiccups getting up and running. My email's petewarden at google.com if you are trying this and hit problems.
  - timanglade 10y ago
    Thanks Pete :)
  - syntaxing 10y ago
    Thanks Tim and Pete!

codesternews 10y ago

which app may i know?

timanglade 10y ago
Not released yet, should go live in April/May!

redsummer 10y ago

How do you embed in a mobile app?

timanglade 10y ago
Essentially there are two ways to do this. The “old” way is to export your TensorFlow neural network into a protobuf file, then load up the TensorFlow interpreter in your iOS/Android app, feed it the neural net, and run the inference directly on device. The GitHub repo [0] has a good set of examples of what that looks like in practice. The new, still experimental way is to compile your neural net into executable code with their XLA / tfcompile tool, and link that into your app. They are adding more docs on this on the TensorFlow website [1]. [0]: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/ios_examples https://github.com/tensorflow/tensorflow/tree/master/tensorf... [1]: https://www.tensorflow.org/versions/master/experimental/xla/tfcompile https://www.tensorflow.org/versions/master/experimental/xla/...
- nojvek 10y ago
  Running neural nets on a standard mobile device will be game changing. I can't wait for mobile devices having custom chips to do AI related tasks. I'm predicting in a decade we'll have offline speech and image recognition running on the phone.
  - xtf 10y ago
    They don't want you to go there. Remind who started the latest big AI Projects (Google, Amazon, Microsoft, Facebook) and I don't think the will stop grabbing data. I think they'll develop a hivemind, where mobile adds to the pool. In short Skynet ;)
    - nojvek 10y ago
      Hopefully the ASICs are fast and open enough so we can see some open AIs who respect our privacy and security concerns