8 ms·

Computing 10,000x more efficiently (2010) [pdf]

91 points by Moshe_Silnorin 11y ago

deleted 11y ago
[deleted]
bra-ket 11y ago
what happened to this project since 2010?
- psilence 11y ago
  Ongoing, by the look of the founder's LinkedIn: https://www.linkedin.com/pub/joseph-bates/2/853/aa3 https://www.linkedin.com/pub/joseph-bates/2/853/aa3 And his company's patents, http://patents.justia.com/assignee/singular-computing-llc http://patents.justia.com/assignee/singular-computing-llc Ah, I get the name now. "The Singularity Is Near" came out in 2005 https://en.wikipedia.org/wiki/The_Singularity_Is_Near https://en.wikipedia.org/wiki/The_Singularity_Is_Near
  - bjd2385 11y ago
    I read that book recently. It was alright, seemed a bit far-fetched to me, however. I mean, obviously we're moving along at a fast pace, but by the time just _some_ of us (let alone _all_) experience the author's projections we'll probably be in the real-life Star Trek era.
jules 11y ago
Unfortunately this solves the wrong problem. The bottleneck isn't arithmetic, it's data movement. The number of transistors doing arithmetic is already a very tiny fraction of a modern chip. Reducing that tiny fraction to an even tinier fraction by making arithmetic inaccurate isn't a good trade-off.
- karmakaze 11y ago
  Which reminds me of the theoretical/thermodynamic perspective of specifically copying data rather than movement. State reduction. https://en.wikipedia.org/wiki/Reversible_computing https://en.wikipedia.org/wiki/Reversible_computing
- _delirium 11y ago
  They seem to be focusing on specific applications where arithmetic might be (or so they hypothesize) a bottleneck. His LinkedIn says, "we recently showed a 6400x improvement in speed/power ratio tracking objects in video for the U.S. Navy".
- minthd 11y ago
  The bottleneck actually is arithmetic. "GPUs have much higher ALU throughput since the GPU chip area is almost entirely ALU" http://devblogs.nvidia.com/parallelforall/bidmach-machine-learning-limit-gpus/ http://devblogs.nvidia.com/parallelforall/bidmach-machine-le... Also on the horizon there is 3d chip manufacturing technology(3d-monolithic) ,with extremely large bandwidth between the two different layers of the chip,possibly being gpu + dram.
  - foobar2020 11y ago
    The bottleneck is often RAM. This is especially clear when writing performance-oriented code in CUDA, where the amount of cores (threads) per one shared memory controller is in the order of thousands.
  - exascale1 11y ago
    The bottleneck is not arithmetic for a long time, it's data movement. Arithmetic is practically free nowadays. See presentation by Horst Simon (Deputy Director of Lawrence Berkeley National Laboratory) "No exascale for you!" [0] The energy cost of transferring a single data word to a distance of 5mm on-chip is higher than the cost of a single FLOP (20 pico-Joules/bit). 5mm =~ the distance to L2 cache or another CPU core. The cost of transferring data off-chip (3D chip and/or RAM) is orders-of-magnitude higher, see graph. [0] http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pdf http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pd...
  - cbsmith 11y ago
    ...but since GPU's already exist, they kind of are the "already at large scale production" solution to the problem. For very little money you can get some pretty insane single precision throughput for SIMD calculations. What you run in to are problems feeding the beast data fast enough.
- modeless 11y ago
  There's a new application area where arithmetic is the bottleneck, and accuracy is less important: Convolutional neural networks. I would love to see a convnet chip built using these techniques.
  - varelse 11y ago
    There's a clear and massive movement of data forward for prediction in a CNN, and all that plus saving intermediate state and then moving the same amount of data backwards when performing backpropagation. But don't believe me, don't believe a word I type. Go prove the above wrong by building a killer AlexNet implementation out of this chip. See slide 9 of this presentation for why I don't think you can do this (too much data transfer): http://www.slideshare.net/embeddedvision/tradeoffs-in-implementing-deep-neural-networks-on-fpgas-a-presentation-from-auviz-systems http://www.slideshare.net/embeddedvision/tradeoffs-in-implem...
    - raverbashing 11y ago
      Slide 9 makes it clear that, for most layers computation takes more time than data transfers If you can reduce the size of the computation units you can have one (or several) for each layer, making it possible to hardwire the transfer between layers
- anon4 11y ago
  What if we take the arithmetic and move it next to the data? Yes, I'm still waiting for memristors.
- alephnil 11y ago
  What have happened the last 20 years is that the total amount of arithmetic that can be done has increased tremendously, while communication has not to the same extent. That means that more problems than before is communication bound (even inside a processor) than before. Since problems range from extremly IO bound to entirely CPU-bound, more problems are bound by communication than before. A friend of mine that worked with supercomputers back in the 90s used to say that a supercomputer was a device that converted a computational problem to a communication problem. Today this is true for mainstream computing as well. There are still problems that can take advantage of the massive amount of computing available (Some mentioned Convolutional Neural Networks), but many tasks that used to be CPU bound is now effectively IO bound.
msandford 11y ago
I might also ask why only go for a 1% number? It seems like it'd be pretty doable to get a 0.1% approximation as that's only a 30dB SNR versus a 20dB SNR. Maybe I'm super naive but it doesn't seem like it should be tremendously difficult even if it does cut your core count by 20-50%. Part of the reason I argue for this is that there are tons of sensors which are 0.1% sensors and if you can offer the rest of the computational pipeline at 0.1% then (so long as your errors don't accumulate)_you don't lose any accuracy processing your information this way. It also seems like this would be pretty great for graphics cards, no? I mean it'd take a lot of work to make OpenGL run on it, but once you did you could have either very inexpensive cards, very powerful cards, or both.
- oh_sigh 11y ago
  > so long as your errors don't accumulate I think that may end up being the hardest part of all this.
  - msandford 11y ago
    Totally agree.
sklogic 11y ago
Many GPUs used to have a tiny, fast and very imprecise SFU - it's ok for GLES but useless for anything else.
- boxfire 11y ago
  Any problem where the data is imprecise or the model itself is very approximate. Machine learning type problems, machine vision, as the paper shows interferometry, lossy image processing. I can think of a few more data processing problems, but I think you might get the point. The use of a precision processor even in single precision when your input data is already 5%+ noise is a huge waste. This has immense application.
  - sklogic 11y ago
    Yes, makes sense... These units are accessible via (undocumented) intrinsics in some OpenCL implementations (or in GLSL), but, unfortunately, there is no portable solution. And the FP precision requirements in the OpenCL standard are way too high, even for the FP16 extension.
Quanticles 11y ago
Our company works on this kind of stuff for those who are interested (http://isosemi.com http://isosemi.com) We're seeing more like 10-100x improvements in energy efficiency and performance, not 10000x, unless the comparison point is a full blown CPU/GPU.
- nickpsecurity 11y ago
  I was recently digging into analog papers to try to figure out how to apply it more general-purpose or at least tap into it for special purpose functions. I'm not hardware trained so much as a systems guy who knows enough to give others tips on what to look into. Here's some links I discovered: https://www.schneier.com/blog/archives/2015/07/friday_squid_bl_488.html#c6701962 https://www.schneier.com/blog/archives/2015/07/friday_squid_... Accidentally running into another group using analog selectively for acceleration is pretty neat. The coprocessor was a believable improvement showing analog power. What's your thoughts on the computing with free space and no transistors stuff? Do those other links come off as bogus to a pro or plausible enough to encourage local college students to try something with it? I think there's vast untapped potential in shifting certain functions back to analog and improving the integration of the two. Maybe in general-purpose, too. Almost certainly in INFOSEC w/ analog supporting obfuscation and tamper-detection.
  - Quanticles 11y ago
    There is a lot of interesting research out there on analog computing, analog neural networks, and far-out stuff like transistor-free computing. Going from research project to product on Digikey is a really huge leap for most research though. Designing a chip is very expensive, so the product better be a slam dunk. Most of these analog neural network projects can do some sort of learning with small black and white patterns, which does not approach the accuracy or scale of software neural networks. What we're working on is an accelerator for the convolutional neural networks that are winning competitions like ILSVRC. Even that by itself is insufficient for a business case, though. You also have to have end application in mind too, and that end application better be power intensive or performance constrained enough that software cannot accomplish what you need it to do. Because, if software is good enough, then why take a risk on a fancy new hardware component?
karmakaze 11y ago
Very interesting. Wondering if the Apple ISA will have anything to do with this.
asgard1024 11y ago
I have wondered why don't we use only exponent in floating point numbers as a representation. If the exponent is close to 1 then you can represent mantissa of any size with it. It seems that the article is suggesting something similar.

deleted 11y ago

[deleted]

bra-ket 11y ago

what happened to this project since 2010?

psilence 11y ago
Ongoing, by the look of the founder's LinkedIn: https://www.linkedin.com/pub/joseph-bates/2/853/aa3 https://www.linkedin.com/pub/joseph-bates/2/853/aa3 And his company's patents, http://patents.justia.com/assignee/singular-computing-llc http://patents.justia.com/assignee/singular-computing-llc Ah, I get the name now. "The Singularity Is Near" came out in 2005 https://en.wikipedia.org/wiki/The_Singularity_Is_Near https://en.wikipedia.org/wiki/The_Singularity_Is_Near
- bjd2385 11y ago
  I read that book recently. It was alright, seemed a bit far-fetched to me, however. I mean, obviously we're moving along at a fast pace, but by the time just _some_ of us (let alone _all_) experience the author's projections we'll probably be in the real-life Star Trek era.

jules 11y ago

Unfortunately this solves the wrong problem. The bottleneck isn't arithmetic, it's data movement. The number of transistors doing arithmetic is already a very tiny fraction of a modern chip. Reducing that tiny fraction to an even tinier fraction by making arithmetic inaccurate isn't a good trade-off.

karmakaze 11y ago
Which reminds me of the theoretical/thermodynamic perspective of specifically copying data rather than movement. State reduction. https://en.wikipedia.org/wiki/Reversible_computing https://en.wikipedia.org/wiki/Reversible_computing
_delirium 11y ago
They seem to be focusing on specific applications where arithmetic might be (or so they hypothesize) a bottleneck. His LinkedIn says, "we recently showed a 6400x improvement in speed/power ratio tracking objects in video for the U.S. Navy".
minthd 11y ago
The bottleneck actually is arithmetic. "GPUs have much higher ALU throughput since the GPU chip area is almost entirely ALU" http://devblogs.nvidia.com/parallelforall/bidmach-machine-learning-limit-gpus/ http://devblogs.nvidia.com/parallelforall/bidmach-machine-le... Also on the horizon there is 3d chip manufacturing technology(3d-monolithic) ,with extremely large bandwidth between the two different layers of the chip,possibly being gpu + dram.
- foobar2020 11y ago
  The bottleneck is often RAM. This is especially clear when writing performance-oriented code in CUDA, where the amount of cores (threads) per one shared memory controller is in the order of thousands.
- exascale1 11y ago
  The bottleneck is not arithmetic for a long time, it's data movement. Arithmetic is practically free nowadays. See presentation by Horst Simon (Deputy Director of Lawrence Berkeley National Laboratory) "No exascale for you!" [0] The energy cost of transferring a single data word to a distance of 5mm on-chip is higher than the cost of a single FLOP (20 pico-Joules/bit). 5mm =~ the distance to L2 cache or another CPU core. The cost of transferring data off-chip (3D chip and/or RAM) is orders-of-magnitude higher, see graph. [0] http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pdf http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pd...
- cbsmith 11y ago
  ...but since GPU's already exist, they kind of are the "already at large scale production" solution to the problem. For very little money you can get some pretty insane single precision throughput for SIMD calculations. What you run in to are problems feeding the beast data fast enough.
modeless 11y ago
There's a new application area where arithmetic is the bottleneck, and accuracy is less important: Convolutional neural networks. I would love to see a convnet chip built using these techniques.
- varelse 11y ago
  There's a clear and massive movement of data forward for prediction in a CNN, and all that plus saving intermediate state and then moving the same amount of data backwards when performing backpropagation. But don't believe me, don't believe a word I type. Go prove the above wrong by building a killer AlexNet implementation out of this chip. See slide 9 of this presentation for why I don't think you can do this (too much data transfer): http://www.slideshare.net/embeddedvision/tradeoffs-in-implementing-deep-neural-networks-on-fpgas-a-presentation-from-auviz-systems http://www.slideshare.net/embeddedvision/tradeoffs-in-implem...
  - raverbashing 11y ago
    Slide 9 makes it clear that, for most layers computation takes more time than data transfers If you can reduce the size of the computation units you can have one (or several) for each layer, making it possible to hardwire the transfer between layers
anon4 11y ago
What if we take the arithmetic and move it next to the data? Yes, I'm still waiting for memristors.
alephnil 11y ago
What have happened the last 20 years is that the total amount of arithmetic that can be done has increased tremendously, while communication has not to the same extent. That means that more problems than before is communication bound (even inside a processor) than before. Since problems range from extremly IO bound to entirely CPU-bound, more problems are bound by communication than before. A friend of mine that worked with supercomputers back in the 90s used to say that a supercomputer was a device that converted a computational problem to a communication problem. Today this is true for mainstream computing as well. There are still problems that can take advantage of the massive amount of computing available (Some mentioned Convolutional Neural Networks), but many tasks that used to be CPU bound is now effectively IO bound.

msandford 11y ago

I might also ask why only go for a 1% number? It seems like it'd be pretty doable to get a 0.1% approximation as that's only a 30dB SNR versus a 20dB SNR. Maybe I'm super naive but it doesn't seem like it should be tremendously difficult even if it does cut your core count by 20-50%. Part of the reason I argue for this is that there are tons of sensors which are 0.1% sensors and if you can offer the rest of the computational pipeline at 0.1% then (so long as your errors don't accumulate)_you don't lose any accuracy processing your information this way. It also seems like this would be pretty great for graphics cards, no? I mean it'd take a lot of work to make OpenGL run on it, but once you did you could have either very inexpensive cards, very powerful cards, or both.

oh_sigh 11y ago
> so long as your errors don't accumulate I think that may end up being the hardest part of all this.
- msandford 11y ago
  Totally agree.

sklogic 11y ago

Many GPUs used to have a tiny, fast and very imprecise SFU - it's ok for GLES but useless for anything else.

boxfire 11y ago
Any problem where the data is imprecise or the model itself is very approximate. Machine learning type problems, machine vision, as the paper shows interferometry, lossy image processing. I can think of a few more data processing problems, but I think you might get the point. The use of a precision processor even in single precision when your input data is already 5%+ noise is a huge waste. This has immense application.
- sklogic 11y ago
  Yes, makes sense... These units are accessible via (undocumented) intrinsics in some OpenCL implementations (or in GLSL), but, unfortunately, there is no portable solution. And the FP precision requirements in the OpenCL standard are way too high, even for the FP16 extension.

Quanticles 11y ago

Our company works on this kind of stuff for those who are interested (http://isosemi.com http://isosemi.com) We're seeing more like 10-100x improvements in energy efficiency and performance, not 10000x, unless the comparison point is a full blown CPU/GPU.

nickpsecurity 11y ago
I was recently digging into analog papers to try to figure out how to apply it more general-purpose or at least tap into it for special purpose functions. I'm not hardware trained so much as a systems guy who knows enough to give others tips on what to look into. Here's some links I discovered: https://www.schneier.com/blog/archives/2015/07/friday_squid_bl_488.html#c6701962 https://www.schneier.com/blog/archives/2015/07/friday_squid_... Accidentally running into another group using analog selectively for acceleration is pretty neat. The coprocessor was a believable improvement showing analog power. What's your thoughts on the computing with free space and no transistors stuff? Do those other links come off as bogus to a pro or plausible enough to encourage local college students to try something with it? I think there's vast untapped potential in shifting certain functions back to analog and improving the integration of the two. Maybe in general-purpose, too. Almost certainly in INFOSEC w/ analog supporting obfuscation and tamper-detection.
- Quanticles 11y ago
  There is a lot of interesting research out there on analog computing, analog neural networks, and far-out stuff like transistor-free computing. Going from research project to product on Digikey is a really huge leap for most research though. Designing a chip is very expensive, so the product better be a slam dunk. Most of these analog neural network projects can do some sort of learning with small black and white patterns, which does not approach the accuracy or scale of software neural networks. What we're working on is an accelerator for the convolutional neural networks that are winning competitions like ILSVRC. Even that by itself is insufficient for a business case, though. You also have to have end application in mind too, and that end application better be power intensive or performance constrained enough that software cannot accomplish what you need it to do. Because, if software is good enough, then why take a risk on a fancy new hardware component?

karmakaze 11y ago

Very interesting. Wondering if the Apple ISA will have anything to do with this.

asgard1024 11y ago

I have wondered why don't we use only exponent in floating point numbers as a representation. If the exponent is close to 1 then you can represent mantissa of any size with it. It seems that the article is suggesting something similar.