8 ms·
Emergent Chip Vastly Accelerates Deep Neural Networks
- jph 11y agoSummary: The chip's impressive speed gains are because it uses on-chip SRAM instead of off-chip DRAM. The chip is able to fit much more into SRAM because the chip uses network compression, pruning, quantization, Huffman encoding, etc.
- 6502nerdface 11y agoAlso, the on-board RAM is SRAM rather than DRAM.
- johnzabroski 11y agoAlso, that this is another example of more specialized chip sets eating away at general purpose CPU/GPU market share.
- sawwit 11y ago> Static random-access memory (static RAM or SRAM) is a type of semiconductor memory that uses bistable latching circuitry (flip-flop) to store each bit. SRAM exhibits data remanence,[1] but it is still volatile in the conventional sense that data is eventually lost when the memory is not powered. The term static differentiates SRAM from DRAM (dynamic random-access memory) which must be periodically refreshed. SRAM is faster and more expensive than DRAM; it is typically used for CPU cache while DRAM is used for a computer's main memory. WP
- dharma1 11y agoHere's the paper - http://arxiv.org/pdf/1510.00149.pdf http://arxiv.org/pdf/1510.00149.pdf The compressed network achieves a decent speedup and energy saving on current hardware too (desktop/mobile) without significant loss of accuracy
- alvern 11y agothank you for sharing
- nl 11y agoIs this the correct paper? It seems to cover the (impressive!) speedups from compression and encoding, but nothing about the EIE chip mentioned.
- dharma1 11y agoYep it only covers the compression/pruning. Here is a talk by the author, covering the chip too -http://web.stanford.edu/class/ee380/Abstracts/160106.html http://web.stanford.edu/class/ee380/Abstracts/160106.html
- jimbokun 11y agoIn the past, specialized architectures tended to lose out to the performance gains of generic x86 chips. I'm thinking RISC vs. CISC, or Sun's attempt at a custom Java-optimized processor. Is that no longer the case? ARM architectures seem to be beating x86 for low power, mobile devices. GPUs are being used for many easily parallelizable workloads. Is an overall slow down in Moore's law making chips designed for specific tasks (like Deep Neural Nets), attractive again?
- emcq 11y agoI'm not sure CISC beat RISC, except for complicating the instruction set enough to keep costs high for competitors. Underneath, Intel processors translate x86 to simpler RISC like microcode. They could make a more efficient chip without this translation component, and probably holds them back a bit in low power stuff.
- cobaltblue 11y agoIt's more like x86 beat everything, which is also saying that externally CISC/RISC didn't matter so much. I like the section on RISC here: http://danluu.com/butler-lampson-1999/ http://danluu.com/butler-lampson-1999/ Specifically part of the last paragraph: "It’s possible to nitpick RISC being a no by saying that modern processors translate x86 ops into RISC micro-ops internally, but if you listened to talk at the time, people thought that having a external RISC ISA would be so much lower overhead that RISC would win, which has clearly not happened. Moreover, modern chips also do micro-op fusion in order to fuse operations into decidedly un-RISC-y operations."
- chongli 11y agox86 beat everything largely due to momentum. Intel had the greatest ability to invest in process shrinking technology and the brute force to engineer their way out of architectural dead-ends (see NetBurst). Now that process shrinking has become extremely difficult, everybody else is starting to catch up.
- p1esk 11y agoYes, it's still the case. Moore's Law is still alive (e.g. Tegra X1 vs K1). However, this paper is not about using specialized architectures, it's about compressing weight matrices and putting them in cache.
- jerf 11y agoIs the embedded case really that interesting? Almost by definition, the embedded device will be receiving a tiny fraction of the data in the world that it may be concerned about. It seems unlikely to me that an embedded, power-constrained device is going to "deep learn" anything all that useful that wouldn't be better learned in something with more data and power available. But I do mean this as a question, if anybody's got a really cool use case in hand. (Please something more specific than text that boils down to "something something sensor network internet of things local conditions something".)
- cscurmudgeon 11y agoTrue. The cases where this would be useful (extremely low training data) there are machine learning methods that are much much better than deep learning, which is a glutton for data comparatively. E.g. handwriting recognition or signature recognition (where you would have only a few samples). http://www.sciencemag.org/content/350/6266/1332.abstract http://www.sciencemag.org/content/350/6266/1332.abstract
- IanCal 11y agoI didn't get the impression this was for training, but for running a trained network. This may lead to high quality speech recognition or image recognition right on the device, without requiring a beast of a GPU.
- cscurmudgeon 11y agoAh ok, got it. But is running that slow that you need special hardware?
- IanCal 11y agoThe large nets, yes. Partly because bigger nets tend to do better so we've built massive nets that run pretty well on big GPUs. But that's the state of things, we have impressive results but from huge neural nets that we really cannot run in realtime on a mobile device. It's particularly bad if you want to run them on a small, battery powered device. Far more so if you only have one image to process, here they're seeing single image speedups of over 10x compared to GPUs (but slower than batched processing) and energy efficiencies about 1000+ times better than mobile GPUs (and far more compared to the beasts in your desktop).
- melted 11y agoMark my words, in the next couple of years we'll see custom silicon that massively improves performance per watt for DNNs by using fixed point, quantization, and saturation arithmetic. The gain in performance per watt will be at least an order of magnitude. This will make DNNs worthwhile in a lot more classification problems where currently they are simply too slow.
- nshm 11y agoMark my words, DNNs are not really the most efficient structure for predicting model due to the "distributed" representation that makes them good predictors, but also makes them hard to train and resource-consuming to apply. In few years DNNs will be replaced by more efficient models.
- melted 11y agoYou do realize you came up with this text using a deep biological neural network, right?
- nshm 11y agoSure, but they aren't necessary the most efficient model possible
- melted 11y agoSeems pretty efficient to me, at least per watt.
- emcq 11y agoPlease don't compare the complexity of the brain to artificial neural networks. The complexity of a single neuron is far more sophisticated than we use for machine learning. Many people are starting to call the machine learning "neurons" as "units" to prevent this comparison.
- dman 11y agoAny pointers to more approaches for more efficient alternatives?
- bbhill 11y agoThese guys need to work on their dataviz. Could have made some of the tables into graphs, really.
- nshm 11y agoThose interested in optimal neural network compression might consider the paper "Bitwise Neural Networks" by Kim and Smaragdis http://paris.cs.illinois.edu/pubs/minje-icmlw2015.pdf http://paris.cs.illinois.edu/pubs/minje-icmlw2015.pdf which enables much better compression than simple quantization and pruning.
- nharada 11y agoHow do you mean "much better compression"? Won't replacing 32bit multiplies by bitwise operations save 32x the memory[1]? Han et al. show not only 35-49x improvement, but on much more difficult benchmarks (MNIST vs Alexnet/VGG). Combining these two techniques would be really cool and if the bitwise network can work with larger, more complex networks like VGG would be a massive game-changer, allowing these nets to fit on almost any device. [1] http://minjekim.com/demo_bnn.html http://minjekim.com/demo_bnn.html
- zhyan7109 11y agoThis will be a game changer a few years down the line! Can't wait to see the commercial SW stack complimenting this.