4 ms·

Very briefly: operations like Transpose, Strided Slice, Padding, and batching are standard operations used to build Convolutional Neural Networks, which is in

Very briefly: operations like Transpose, Strided Slice, Padding, and batching are standard operations used to build Convolutional Neural Networks, which is in turn is a standard building block for a vast array of machine learning applications (eg, image recognition)

nathanaldensr 9y ago
Ah, cool. So is the goal with having this tech on the phone that the phone won't need to communicate with servers to do this sort of processing? I'm struggling to think of a use case for this.
- cdancette 9y ago
  Yes, I think that's exactly the case, for doing object / face recognition from your phone camera in real time for example. Or maybe on device speech recognition in the future
- spyhi 9y ago
  > the goal with having this tech on the phone that the phone won't need to communicate with servers to do this sort of processing? Yeah, that's basically it. In short, the operations listed are pretty computationally intensive on a CPU, but pretty easy to parallelize for the data structures that are commonly used in machine learning (matrices and tensors). I presume that, by creating an API, Google can abstract out these common ML operations to work on the most efficient hardware on the device, whether a GPU or custom ML hardware such as Apple's Neural Engine. In theory, this should make on-device machine learning and inference more power-efficient, and could reduce reliance on server communication, improving privacy and probably making lightweight ML applications feel faster.
  - richdougherty 9y ago
    > custom ML hardware I think it's speculation still, but it seems likely that the Pixel Visual Core chip included in Pixel 2s could be a target for the API. https://arstechnica.com/gadgets/2017/10/google-launches-the-android-8-1-developer-preview/ https://arstechnica.com/gadgets/2017/10/google-launches-the-... The chip is already a target for some ML APIs. https://blog.google/products/pixel/pixel-visual-core-image-processing-and-machine-learning-pixel-2/ https://blog.google/products/pixel/pixel-visual-core-image-p...
- nl 9y ago
  The new YouTube background removal feature is done on-device using neural networks. https://research.googleblog.com/2018/03/mobile-real-time-video-segmentation.html https://research.googleblog.com/2018/03/mobile-real-time-vid...

nathanaldensr 9y ago

Ah, cool. So is the goal with having this tech on the phone that the phone won't need to communicate with servers to do this sort of processing? I'm struggling to think of a use case for this.

cdancette 9y ago
Yes, I think that's exactly the case, for doing object / face recognition from your phone camera in real time for example. Or maybe on device speech recognition in the future
spyhi 9y ago
> the goal with having this tech on the phone that the phone won't need to communicate with servers to do this sort of processing? Yeah, that's basically it. In short, the operations listed are pretty computationally intensive on a CPU, but pretty easy to parallelize for the data structures that are commonly used in machine learning (matrices and tensors). I presume that, by creating an API, Google can abstract out these common ML operations to work on the most efficient hardware on the device, whether a GPU or custom ML hardware such as Apple's Neural Engine. In theory, this should make on-device machine learning and inference more power-efficient, and could reduce reliance on server communication, improving privacy and probably making lightweight ML applications feel faster.
- richdougherty 9y ago
  > custom ML hardware I think it's speculation still, but it seems likely that the Pixel Visual Core chip included in Pixel 2s could be a target for the API. https://arstechnica.com/gadgets/2017/10/google-launches-the-android-8-1-developer-preview/ https://arstechnica.com/gadgets/2017/10/google-launches-the-... The chip is already a target for some ML APIs. https://blog.google/products/pixel/pixel-visual-core-image-processing-and-machine-learning-pixel-2/ https://blog.google/products/pixel/pixel-visual-core-image-p...
nl 9y ago
The new YouTube background removal feature is done on-device using neural networks. https://research.googleblog.com/2018/03/mobile-real-time-video-segmentation.html https://research.googleblog.com/2018/03/mobile-real-time-vid...