4 ms·

What do you mean by expensive? Smaller neural networks are computationally cheap to run forward propagation on, which is all you need to do after deployment.

What do you mean by expensive? Smaller neural networks are computationally cheap to run forward propagation on, which is all you need to do after deployment. This model is running in real time on your phone, after all. I'm also confused by your suggestion to use logistic regression instead. The accuracy will certainly be worse - even if you use relatively state of the art non-convolutional features like a spatial pyramid with SIFT or HOG (the computation of these features would almost certainly be more expensive than running forward propagation for a reasonably sized neural network, in any case). Thoroughly confused over here.

YeGoblynQueenne 10y ago
Both bits you're confused about, how deep nets are expensive to run and logistic regression scales better etc, are mentioned in the article and I use them in the same context. EDIT: can't wait for the post timer to cool down, so please keep in mind I mean the submitted article, not the google research blog cited in that article.
- nl 10y ago
  how deep nets are expensive to run I think you are overstating what the article says here. Google Translate lets you view an image through your phone's camera and it replaces - in real time - words in one language with the translated version in another. It's pretty amazing, and to get it to work the whole stack needs to be optimized. Part of that is making the network as efficient as possible, and one way to do that is to make it smaller. logistic regression scales better etc, are mentioned in the article I'm a bit confused. Neither of the words "logistic" or "regression" are mentioned in the article you linked to (http://googleresearch.blogspot.com.au/2015/07/how-google-translate-squeezes-deep.html http://googleresearch.blogspot.com.au/2015/07/how-google-tra...).
  - YeGoblynQueenne 10y ago
    >> I'm a bit confused. Neither of the words "logistic" or "regression" are mentioned in the article you linked to As noted above, logistic regression as a more scalable alternative to deep neural networks is mentioned in the parent article, not the link to the google research blog. The google research blog is linked to from the parent article as proof of the parent article's claim that deep nets do machine translation on your phone.

YeGoblynQueenne 10y ago

Both bits you're confused about, how deep nets are expensive to run and logistic regression scales better etc, are mentioned in the article and I use them in the same context. EDIT: can't wait for the post timer to cool down, so please keep in mind I mean the submitted article, not the google research blog cited in that article.

nl 10y ago
how deep nets are expensive to run I think you are overstating what the article says here. Google Translate lets you view an image through your phone's camera and it replaces - in real time - words in one language with the translated version in another. It's pretty amazing, and to get it to work the whole stack needs to be optimized. Part of that is making the network as efficient as possible, and one way to do that is to make it smaller. logistic regression scales better etc, are mentioned in the article I'm a bit confused. Neither of the words "logistic" or "regression" are mentioned in the article you linked to (http://googleresearch.blogspot.com.au/2015/07/how-google-translate-squeezes-deep.html http://googleresearch.blogspot.com.au/2015/07/how-google-tra...).
- YeGoblynQueenne 10y ago
  >> I'm a bit confused. Neither of the words "logistic" or "regression" are mentioned in the article you linked to As noted above, logistic regression as a more scalable alternative to deep neural networks is mentioned in the parent article, not the link to the google research blog. The google research blog is linked to from the parent article as proof of the parent article's claim that deep nets do machine translation on your phone.