5 ms·

Unfortunately this solves the wrong problem. The bottleneck isn't arithmetic, it's data movement. The number of transistors doing arithmetic is already a very t

by jules 11y ago

Unfortunately this solves the wrong problem. The bottleneck isn't arithmetic, it's data movement. The number of transistors doing arithmetic is already a very tiny fraction of a modern chip. Reducing that tiny fraction to an even tinier fraction by making arithmetic inaccurate isn't a good trade-off.

karmakaze 11y ago
Which reminds me of the theoretical/thermodynamic perspective of specifically copying data rather than movement. State reduction. https://en.wikipedia.org/wiki/Reversible_computing https://en.wikipedia.org/wiki/Reversible_computing
_delirium 11y ago
They seem to be focusing on specific applications where arithmetic might be (or so they hypothesize) a bottleneck. His LinkedIn says, "we recently showed a 6400x improvement in speed/power ratio tracking objects in video for the U.S. Navy".
minthd 11y ago
The bottleneck actually is arithmetic. "GPUs have much higher ALU throughput since the GPU chip area is almost entirely ALU" http://devblogs.nvidia.com/parallelforall/bidmach-machine-learning-limit-gpus/ http://devblogs.nvidia.com/parallelforall/bidmach-machine-le... Also on the horizon there is 3d chip manufacturing technology(3d-monolithic) ,with extremely large bandwidth between the two different layers of the chip,possibly being gpu + dram.
- foobar2020 11y ago
  The bottleneck is often RAM. This is especially clear when writing performance-oriented code in CUDA, where the amount of cores (threads) per one shared memory controller is in the order of thousands.
- exascale1 11y ago
  The bottleneck is not arithmetic for a long time, it's data movement. Arithmetic is practically free nowadays. See presentation by Horst Simon (Deputy Director of Lawrence Berkeley National Laboratory) "No exascale for you!" [0] The energy cost of transferring a single data word to a distance of 5mm on-chip is higher than the cost of a single FLOP (20 pico-Joules/bit). 5mm =~ the distance to L2 cache or another CPU core. The cost of transferring data off-chip (3D chip and/or RAM) is orders-of-magnitude higher, see graph. [0] http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pdf http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pd...
- cbsmith 11y ago
  ...but since GPU's already exist, they kind of are the "already at large scale production" solution to the problem. For very little money you can get some pretty insane single precision throughput for SIMD calculations. What you run in to are problems feeding the beast data fast enough.
modeless 11y ago
There's a new application area where arithmetic is the bottleneck, and accuracy is less important: Convolutional neural networks. I would love to see a convnet chip built using these techniques.
- varelse 11y ago
  There's a clear and massive movement of data forward for prediction in a CNN, and all that plus saving intermediate state and then moving the same amount of data backwards when performing backpropagation. But don't believe me, don't believe a word I type. Go prove the above wrong by building a killer AlexNet implementation out of this chip. See slide 9 of this presentation for why I don't think you can do this (too much data transfer): http://www.slideshare.net/embeddedvision/tradeoffs-in-implementing-deep-neural-networks-on-fpgas-a-presentation-from-auviz-systems http://www.slideshare.net/embeddedvision/tradeoffs-in-implem...
  - raverbashing 11y ago
    Slide 9 makes it clear that, for most layers computation takes more time than data transfers If you can reduce the size of the computation units you can have one (or several) for each layer, making it possible to hardwire the transfer between layers
anon4 11y ago
What if we take the arithmetic and move it next to the data? Yes, I'm still waiting for memristors.
alephnil 11y ago
What have happened the last 20 years is that the total amount of arithmetic that can be done has increased tremendously, while communication has not to the same extent. That means that more problems than before is communication bound (even inside a processor) than before. Since problems range from extremly IO bound to entirely CPU-bound, more problems are bound by communication than before. A friend of mine that worked with supercomputers back in the 90s used to say that a supercomputer was a device that converted a computational problem to a communication problem. Today this is true for mainstream computing as well. There are still problems that can take advantage of the massive amount of computing available (Some mentioned Convolutional Neural Networks), but many tasks that used to be CPU bound is now effectively IO bound.

by jules 11y ago

karmakaze 11y ago

Which reminds me of the theoretical/thermodynamic perspective of specifically copying data rather than movement. State reduction. https://en.wikipedia.org/wiki/Reversible_computing https://en.wikipedia.org/wiki/Reversible_computing

_delirium 11y ago

They seem to be focusing on specific applications where arithmetic might be (or so they hypothesize) a bottleneck. His LinkedIn says, "we recently showed a 6400x improvement in speed/power ratio tracking objects in video for the U.S. Navy".

minthd 11y ago

The bottleneck actually is arithmetic. "GPUs have much higher ALU throughput since the GPU chip area is almost entirely ALU" http://devblogs.nvidia.com/parallelforall/bidmach-machine-learning-limit-gpus/ http://devblogs.nvidia.com/parallelforall/bidmach-machine-le... Also on the horizon there is 3d chip manufacturing technology(3d-monolithic) ,with extremely large bandwidth between the two different layers of the chip,possibly being gpu + dram.

foobar2020 11y ago
The bottleneck is often RAM. This is especially clear when writing performance-oriented code in CUDA, where the amount of cores (threads) per one shared memory controller is in the order of thousands.
exascale1 11y ago
The bottleneck is not arithmetic for a long time, it's data movement. Arithmetic is practically free nowadays. See presentation by Horst Simon (Deputy Director of Lawrence Berkeley National Laboratory) "No exascale for you!" [0] The energy cost of transferring a single data word to a distance of 5mm on-chip is higher than the cost of a single FLOP (20 pico-Joules/bit). 5mm =~ the distance to L2 cache or another CPU core. The cost of transferring data off-chip (3D chip and/or RAM) is orders-of-magnitude higher, see graph. [0] http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pdf http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pd...
cbsmith 11y ago
...but since GPU's already exist, they kind of are the "already at large scale production" solution to the problem. For very little money you can get some pretty insane single precision throughput for SIMD calculations. What you run in to are problems feeding the beast data fast enough.

modeless 11y ago

There's a new application area where arithmetic is the bottleneck, and accuracy is less important: Convolutional neural networks. I would love to see a convnet chip built using these techniques.

varelse 11y ago
There's a clear and massive movement of data forward for prediction in a CNN, and all that plus saving intermediate state and then moving the same amount of data backwards when performing backpropagation. But don't believe me, don't believe a word I type. Go prove the above wrong by building a killer AlexNet implementation out of this chip. See slide 9 of this presentation for why I don't think you can do this (too much data transfer): http://www.slideshare.net/embeddedvision/tradeoffs-in-implementing-deep-neural-networks-on-fpgas-a-presentation-from-auviz-systems http://www.slideshare.net/embeddedvision/tradeoffs-in-implem...
- raverbashing 11y ago
  Slide 9 makes it clear that, for most layers computation takes more time than data transfers If you can reduce the size of the computation units you can have one (or several) for each layer, making it possible to hardwire the transfer between layers

anon4 11y ago

What if we take the arithmetic and move it next to the data? Yes, I'm still waiting for memristors.

alephnil 11y ago

What have happened the last 20 years is that the total amount of arithmetic that can be done has increased tremendously, while communication has not to the same extent. That means that more problems than before is communication bound (even inside a processor) than before. Since problems range from extremly IO bound to entirely CPU-bound, more problems are bound by communication than before. A friend of mine that worked with supercomputers back in the 90s used to say that a supercomputer was a device that converted a computational problem to a communication problem. Today this is true for mainstream computing as well. There are still problems that can take advantage of the massive amount of computing available (Some mentioned Convolutional Neural Networks), but many tasks that used to be CPU bound is now effectively IO bound.