4 ms·
Is the embedded case really that interesting? Almost by definition, the embedded device will be receiving a tiny fraction of the data in the world that it may b
by jerf 11y ago
Is the embedded case really that interesting? Almost by definition, the embedded device will be receiving a tiny fraction of the data in the world that it may be concerned about. It seems unlikely to me that an embedded, power-constrained device is going to "deep learn" anything all that useful that wouldn't be better learned in something with more data and power available. But I do mean this as a question, if anybody's got a really cool use case in hand. (Please something more specific than text that boils down to "something something sensor network internet of things local conditions something".)
- cscurmudgeon 11y agoTrue. The cases where this would be useful (extremely low training data) there are machine learning methods that are much much better than deep learning, which is a glutton for data comparatively. E.g. handwriting recognition or signature recognition (where you would have only a few samples). http://www.sciencemag.org/content/350/6266/1332.abstract http://www.sciencemag.org/content/350/6266/1332.abstract
- IanCal 11y agoI didn't get the impression this was for training, but for running a trained network. This may lead to high quality speech recognition or image recognition right on the device, without requiring a beast of a GPU.
- cscurmudgeon 11y agoAh ok, got it. But is running that slow that you need special hardware?
- IanCal 11y agoThe large nets, yes. Partly because bigger nets tend to do better so we've built massive nets that run pretty well on big GPUs. But that's the state of things, we have impressive results but from huge neural nets that we really cannot run in realtime on a mobile device. It's particularly bad if you want to run them on a small, battery powered device. Far more so if you only have one image to process, here they're seeing single image speedups of over 10x compared to GPUs (but slower than batched processing) and energy efficiencies about 1000+ times better than mobile GPUs (and far more compared to the beasts in your desktop).
- nl 11y agoBut that's the state of things, we have impressive results but from huge neural nets that we really cannot run in realtime on a mobile device. This is absolutely incorrect. A mobile device can execute a pretrained model fine. See, for example [1][2][3]. Google's new TensorFlow NN system is explicitly designed to be able to run on mobile devices and comes with a pre-trained image classification NN that workd fine on mobile devices. This doesn't mean that energy saving is unimportant on a mobile device of course. But there are very widely deployed production systems (eg all of Android) that use them now with no special GPU acceleration. [1] http://googleresearch.blogspot.com.au/2015/07/how-google-translate-squeezes-deep.html http://googleresearch.blogspot.com.au/2015/07/how-google-tra... [2] http://googleresearch.blogspot.com.au/2015/09/google-voice-search-faster-and-more.html http://googleresearch.blogspot.com.au/2015/09/google-voice-s... [3] http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41176.pdf http://static.googleusercontent.com/media/research.google.co...
- versteegen 11y agoWell, it depends on the neural network you want to use. At computer vision/image processing conferences I find that it is much more common for people to use preexisting DNNs and plug them into their systems rather than train their own (why would you waste weeks or months on that?). The good pretrained DNNs available today tend to be huge, requiring a large amount of video RAM. I've talked to people writing realtime image processing software (GPU acceleration required) intended for handheld devices which can't yet actually run on the device because they don't have a suitable DNN which will fit in VRAM.
- dharma1 11y agomxnet has some optimised models for mobile devices - http://mxnet.readthedocs.org/en/latest/tutorial/smart_device.html http://mxnet.readthedocs.org/en/latest/tutorial/smart_device...
- nl 11y ago3.5% top-5 error rate on ImageNet running on Android. Hard to beat that (unless you are Microsoft Research Asia). https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android https://github.com/tensorflow/tensorflow/tree/master/tensorf...
- secondtimeuse 11y agoPerforming inference with deep learning modes is computationally costly as well. Its a reason why NVidia has two streams of business, Tesla / Titan / upcoming Volta GPUs to train models and Tegra boards to enable near real time inference.
- jessep 11y agoThis isn't for training, is it? It's for using the results of training immediately (inferring something), without the need for a network round trip (as far as I understand it). So, you might still send the request to the network to continue training the model, but by the time you do, your answer has already computed on the local machine for local consumption.