3 ms·

One has to distinguish between training and inference when talking about "machine learning". Training a model is a long and and resource intensive process, even

One has to distinguish between training and inference when talking about "machine learning". Training a model is a long and and resource intensive process, even if transfer learning is used. Inference is much less energy intensive and could be done on small chips. Regardless, I'm not as certain as the author about the future of ML on small devices. Some ML models are huge and needs to be updated frequently, therefore there is little sense in downloading those to small devices. In such cases, it makes much more sense to send feature data to a remote server that can generate a prediction within milliseconds, and then transmit that prediction back to the device.

blt 8y ago
There has been a lot of research into using very low bit depth weights in neural nets, pruning, etc. I am pretty confident that this research, combined with purpose-designed silicon, will allow us to evaluate quite powerful neural net models on embedded systems.
okket 8y ago
> In a lot of cases, it makes much more sense to send feature data to a remote server that can generate a prediction within milliseconds, and then transmit that prediction back to the device. It may make sense now, but not having to power up the radio for every decision is a huge gain, as lined out in the article. The current model of dumb (as in ML) devices is coming to an end, see also CoreML from Apple.
adamdrake 8y ago
Good point on the fit/predict difference. However, there are some models and techniques (e.g. logistic regression with hashing trick) where the fit and predict steps aren't all that different: https://adamdrake.com/big-data-small-machine.html https://adamdrake.com/big-data-small-machine.html A big benefit for doing everything on-device is that a lot of privacy concerns can be mitigated. I also agree that sending data to a server for learning is an option, and the privacy problems can be addressed with something like client-side feature hashing as I mention in: https://adamdrake.com/scalable-machine-learning-with-fully-anonymized-data.html https://adamdrake.com/scalable-machine-learning-with-fully-a... However, doing that in a very power-conscious environment dose pose difficulties with radio usage, which is comparatively power hungry. It's probably a case-by-case situation.
smrtinsert 8y ago
The more I learn about machine learning, the more I learn its really all training. After training and a model is available, it seems ready to be commoditized to me. ML as a service seems the only reasonable way the industry can evolve.
- otterley 8y ago
  And it's coming: https://lobe.ai https://lobe.ai

blt 8y ago

There has been a lot of research into using very low bit depth weights in neural nets, pruning, etc. I am pretty confident that this research, combined with purpose-designed silicon, will allow us to evaluate quite powerful neural net models on embedded systems.

okket 8y ago

> In a lot of cases, it makes much more sense to send feature data to a remote server that can generate a prediction within milliseconds, and then transmit that prediction back to the device. It may make sense now, but not having to power up the radio for every decision is a huge gain, as lined out in the article. The current model of dumb (as in ML) devices is coming to an end, see also CoreML from Apple.

adamdrake 8y ago

Good point on the fit/predict difference. However, there are some models and techniques (e.g. logistic regression with hashing trick) where the fit and predict steps aren't all that different: https://adamdrake.com/big-data-small-machine.html https://adamdrake.com/big-data-small-machine.html A big benefit for doing everything on-device is that a lot of privacy concerns can be mitigated. I also agree that sending data to a server for learning is an option, and the privacy problems can be addressed with something like client-side feature hashing as I mention in: https://adamdrake.com/scalable-machine-learning-with-fully-anonymized-data.html https://adamdrake.com/scalable-machine-learning-with-fully-a... However, doing that in a very power-conscious environment dose pose difficulties with radio usage, which is comparatively power hungry. It's probably a case-by-case situation.

smrtinsert 8y ago

The more I learn about machine learning, the more I learn its really all training. After training and a model is available, it seems ready to be commoditized to me. ML as a service seems the only reasonable way the industry can evolve.

otterley 8y ago
And it's coming: https://lobe.ai https://lobe.ai