4 ms·

The main competitor for this is other dedicated neural network hardware, such as the Google TPU: https://cloud.google.com/blog/big-data/2017/05/an-in-depth-look

The main competitor for this is other dedicated neural network hardware, such as the Google TPU: https://cloud.google.com/blog/big-data/2017/05/an-in-depth-look-at-googles-first-tensor-processing-unit-tpu https://cloud.google.com/blog/big-data/2017/05/an-in-depth-l... One potential advantage is that neural network weights could be as small as 8 bits. If you're working with 8-bit quantities, you're saving in terms of memory bandwidth, power, and you can fit more computational units. Neural network hardware also isn't going to need hardware implementations of things like sine/cosine, square root. Can potentially avoid control flow and branching hardware (or have less of it). May not need the whole complex register file and thread swapping machinery that GPUs have.

dragontamer 9y ago
Flexpoint looks like its 16-bits however: https://www.intelnervana.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.intelnervana.com/nervana-engine-delivers-deep-le... And I'm not really sure that stripping "Sin" and "Cos" out of the FPU units would really save that much power and/or improve efficiency. Generally speaking: the biggest problem today is the memory architecture of your computing device (be it a CPU, GPU, Paralella, or FPGA). Efficient CPU programming requires deep understanding of L1 / L2 / L3 / Off-chip cache. And similarly, efficient GPU code require deep understanding of the memory hierarchy. ------------------- https://www.intelnervana.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.intelnervana.com/nervana-engine-delivers-deep-le... Looking at the Nervana posts... it looks like the "memory" question is: 1. HBM: Except both AMD Fury and AMD Vega64 are already using HBM. NVidia also has an HBM offering at the very very high end. 2. 16-bit Flexpoint: Except NVidia and AMD cards already can efficiently calculate half-floats (16-bit floats). 3. MAC: Multiply and Accumulate. AMD and NVidia cards have been optimizing this instruction and can perform Matrix Multiplications at ludicrous speeds already. Maybe the Google TPU can be a theoretical competitor... but until Google's TPU becomes available from newegg.com, it really isn't a consideration for most people. ---------------- The main "advantage" of Nervana looks like its improved memory architecture with "separate pipelines for computation and data management". (aka: vague marketing talk). And I really do think this is the key: depending on how memory is architected on the Nervana system, it really can make-or-break the speed of applications. But is also the part that's the most mysterious with the least amount of documentation so far...
- tachyonbeam 9y ago
  Google plans to sell the TPU as a cloud service. It's also their secret weapon, so to speak. They don't need people to buy it in order for it to be an asset, they mostly want to give themselves an advantage. That is, all the people at DeepMind now have access to faster hardware with more memory. They can run bigger experiments than everyone else. As for memory hierarchy, in hardware, there is always something to gain by specialization. Something like a convolution in a CNN is going to have very predictable memory access patterns. It's possible to take advantage of that. Google claims to already have about 10x speedups over GPUs IIRC.

dragontamer 9y ago

Flexpoint looks like its 16-bits however: https://www.intelnervana.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.intelnervana.com/nervana-engine-delivers-deep-le... And I'm not really sure that stripping "Sin" and "Cos" out of the FPU units would really save that much power and/or improve efficiency. Generally speaking: the biggest problem today is the memory architecture of your computing device (be it a CPU, GPU, Paralella, or FPGA). Efficient CPU programming requires deep understanding of L1 / L2 / L3 / Off-chip cache. And similarly, efficient GPU code require deep understanding of the memory hierarchy. ------------------- https://www.intelnervana.com/nervana-engine-delivers-deep-learning-at-ludicrous-speed/ https://www.intelnervana.com/nervana-engine-delivers-deep-le... Looking at the Nervana posts... it looks like the "memory" question is: 1. HBM: Except both AMD Fury and AMD Vega64 are already using HBM. NVidia also has an HBM offering at the very very high end. 2. 16-bit Flexpoint: Except NVidia and AMD cards already can efficiently calculate half-floats (16-bit floats). 3. MAC: Multiply and Accumulate. AMD and NVidia cards have been optimizing this instruction and can perform Matrix Multiplications at ludicrous speeds already. Maybe the Google TPU can be a theoretical competitor... but until Google's TPU becomes available from newegg.com, it really isn't a consideration for most people. ---------------- The main "advantage" of Nervana looks like its improved memory architecture with "separate pipelines for computation and data management". (aka: vague marketing talk). And I really do think this is the key: depending on how memory is architected on the Nervana system, it really can make-or-break the speed of applications. But is also the part that's the most mysterious with the least amount of documentation so far...

tachyonbeam 9y ago
Google plans to sell the TPU as a cloud service. It's also their secret weapon, so to speak. They don't need people to buy it in order for it to be an asset, they mostly want to give themselves an advantage. That is, all the people at DeepMind now have access to faster hardware with more memory. They can run bigger experiments than everyone else. As for memory hierarchy, in hardware, there is always something to gain by specialization. Something like a convolution in a CNN is going to have very predictable memory access patterns. It's possible to take advantage of that. Google claims to already have about 10x speedups over GPUs IIRC.