4 ms·

The Next Wave of Deep Learning Hardware Architectures

84 points by Katydid 10y ago

alistproducer2 10y ago
Is there anyone who can put into practical terms what these domain-specific processors mean to the future of AI/deep learning?
- dharma1 10y ago
  Out of the ones mentioned in the article Movidius was/is specifically focused on low power vision tasks, the rest are fundamentally same stuff as we have today with CPUs/GPUs, just faster, lower power. Wave Computing will be interesting if it's not vaporware. I posted some more details about them a few weeks ago - https://news.ycombinator.com/item?id=12458171 https://news.ycombinator.com/item?id=12458171
- bmh100 10y ago
  A few of the basic concepts: Smarter memory design means that data does not need to be constantly reloaded, saving time that would otherwise be wasted on data transfer. Using lower-precision, fixed-point arithmetic is more efficient than standard-precision floating-point arithmetic. As a result, less time and components are wasted on unnecessary precision. Someone more knowledgeable about processors can better describe the benefits of the Wave architecture. In terms of benchmarks, Wave claims[1] that a single Wave machine is 25x more efficient than 100 GPUs for Google's Inception network. [1]: http://wavecomp.com/technology/ http://wavecomp.com/technology/
  - varelse 10y ago
    100 of which GPU? And how were they connected to each other? They almost never give enough details to really understand these bold claims. If this is 100 K40s in a 10 Gb/s datacenter, this is old hat. If this is 100 Titan XPs in Big Sur boxes connected by 100+ Gb/s Infiniband, it's really interesting. I doubt it though. I suspect the only immediate threat to NVIDIA is underestimating AMD GPUs. 2+ years out, who knows?
    - bmh100 10y ago
      Wave seems to be referencing this[1] Google Research blog post. I couldn't find any reference to what GPU cards Google uses. It still provides a basis of comparison to Google Cloud at least. But like you say, a specialized data center deployment probably wouldn't show such a dramatic difference. [1]: https://research.googleblog.com/2016/04/announcing-tensorflow-08-now-with.html https://research.googleblog.com/2016/04/announcing-tensorflo...
- emcq 10y ago
  Smarter devices. Better voice recognition, smarter cameras, smarter cars, perhaps smarter medical devices. The GPU is hard to beat for training purposes unless you're Google scale, but not acceptable for some of these lower power inference scenarios. You can't reasonably make a product with a GPU today under 10W.
vonnik_2 10y ago
The article is a little dated. It was written in the wake of Intel acquiring Nervana and Movidius, which it used as a hook to talk about Wave Computing. Wave was founded a few months before Nervana in 2014, and raised a similar amount of money (~$24M), which is just enough to get to your first chip if you don't waste resources. There are other companies tackling this (Cerberas Sys.) and other technologies that can get you acceleration (FPGAs).
- tlarkworthy 10y ago
  And the elephant in the room is Google's chip https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html?m=1 https://cloudplatform.googleblog.com/2016/05/Google-supercha...
WalterBright 10y ago
I'd like to see AI applied to handwritten letters. (I have thousands of them, and want to transcribe them.)
- fspacef 10y ago
  I recently tried to do this with Acrobat Pro with varying degrees of success, you might want to give that a shot if you can get access to the program (slightly cost prohibitive).
  - WalterBright 10y ago
    I wish I could try various ones, but as you say, cost is an issue. I do have some OCRs, and while they do a credible job with printed text (95%), they fail miserably with typewritten letters (75%) (the blotchiness of a mechanical typewriter throws it off) and fail 100% with handwritten letters, even carefully done block letters. I know there is success with domain specific handwriting recognition, like envelope addresses, but nothing that can handle a cursive letter. You'd think with all the facial recognition progress this should be a much simpler problem. (To do a good OCR, it'd have to examine the context of a muddled character to determine what it should be, just like a human does when reading cursive. This should be an ideal candidate for "deep learning".)
    - fspacef 10y ago
      While I think your situation is unique, it would be nice if there was an out-of-the-box ML system that could be trained by individuals for any specific use case (such as transcribing letters). In the future, I think this might be the case where you "teach" a computer or program to recognize and do tasks the same way you would a child or pet. PS- Transcribing letters brings to mind an app on the iPhone that I used to use for instant language translation. Can't quite remember what it was called but Google ended up acquiring them for their OCR technology.
      - WalterBright 10y ago

84 points by Katydid 10y ago

Is there anyone who can put into practical terms what these domain-specific processors mean to the future of AI/deep learning?

dharma1 10y ago
Out of the ones mentioned in the article Movidius was/is specifically focused on low power vision tasks, the rest are fundamentally same stuff as we have today with CPUs/GPUs, just faster, lower power. Wave Computing will be interesting if it's not vaporware. I posted some more details about them a few weeks ago - https://news.ycombinator.com/item?id=12458171 https://news.ycombinator.com/item?id=12458171
bmh100 10y ago
A few of the basic concepts: Smarter memory design means that data does not need to be constantly reloaded, saving time that would otherwise be wasted on data transfer. Using lower-precision, fixed-point arithmetic is more efficient than standard-precision floating-point arithmetic. As a result, less time and components are wasted on unnecessary precision. Someone more knowledgeable about processors can better describe the benefits of the Wave architecture. In terms of benchmarks, Wave claims[1] that a single Wave machine is 25x more efficient than 100 GPUs for Google's Inception network. [1]: http://wavecomp.com/technology/ http://wavecomp.com/technology/
- varelse 10y ago
  100 of which GPU? And how were they connected to each other? They almost never give enough details to really understand these bold claims. If this is 100 K40s in a 10 Gb/s datacenter, this is old hat. If this is 100 Titan XPs in Big Sur boxes connected by 100+ Gb/s Infiniband, it's really interesting. I doubt it though. I suspect the only immediate threat to NVIDIA is underestimating AMD GPUs. 2+ years out, who knows?
  - bmh100 10y ago
    Wave seems to be referencing this[1] Google Research blog post. I couldn't find any reference to what GPU cards Google uses. It still provides a basis of comparison to Google Cloud at least. But like you say, a specialized data center deployment probably wouldn't show such a dramatic difference. [1]: https://research.googleblog.com/2016/04/announcing-tensorflow-08-now-with.html https://research.googleblog.com/2016/04/announcing-tensorflo...
emcq 10y ago
Smarter devices. Better voice recognition, smarter cameras, smarter cars, perhaps smarter medical devices. The GPU is hard to beat for training purposes unless you're Google scale, but not acceptable for some of these lower power inference scenarios. You can't reasonably make a product with a GPU today under 10W.

vonnik_2 10y ago

The article is a little dated. It was written in the wake of Intel acquiring Nervana and Movidius, which it used as a hook to talk about Wave Computing. Wave was founded a few months before Nervana in 2014, and raised a similar amount of money (~$24M), which is just enough to get to your first chip if you don't waste resources. There are other companies tackling this (Cerberas Sys.) and other technologies that can get you acceleration (FPGAs).

tlarkworthy 10y ago
And the elephant in the room is Google's chip https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html?m=1 https://cloudplatform.googleblog.com/2016/05/Google-supercha...

WalterBright 10y ago

I'd like to see AI applied to handwritten letters. (I have thousands of them, and want to transcribe them.)

fspacef 10y ago
I recently tried to do this with Acrobat Pro with varying degrees of success, you might want to give that a shot if you can get access to the program (slightly cost prohibitive).
- WalterBright 10y ago
  I wish I could try various ones, but as you say, cost is an issue. I do have some OCRs, and while they do a credible job with printed text (95%), they fail miserably with typewritten letters (75%) (the blotchiness of a mechanical typewriter throws it off) and fail 100% with handwritten letters, even carefully done block letters. I know there is success with domain specific handwriting recognition, like envelope addresses, but nothing that can handle a cursive letter. You'd think with all the facial recognition progress this should be a much simpler problem. (To do a good OCR, it'd have to examine the context of a muddled character to determine what it should be, just like a human does when reading cursive. This should be an ideal candidate for "deep learning".)
  - fspacef 10y ago
    While I think your situation is unique, it would be nice if there was an out-of-the-box ML system that could be trained by individuals for any specific use case (such as transcribing letters). In the future, I think this might be the case where you "teach" a computer or program to recognize and do tasks the same way you would a child or pet. PS- Transcribing letters brings to mind an app on the iPhone that I used to use for instant language translation. Can't quite remember what it was called but Google ended up acquiring them for their OCR technology.
    - WalterBright 10y ago