8 ms·

Intel Introduces Neuromorphic Test Chip

135 points by 40acres 9y ago

meri_dian 9y ago
>"Researchers have demonstrated learning at a rate that is a 1 million times improvement compared with other typical spiking neural nets as measured by total operations to achieve a given accuracy when solving MNIST digit recognition problems." >"Further, it is up to 1,000 times more energy-efficient than general purpose computing required for typical training systems." Wow. That is tremendous.
- zitterbewegung 9y ago
  Comparing a system to MNIST isn't really an indicator that it provides an actual improvement. Comparing it to VGG would be a better indicator.
  - p1esk 9y ago
    MNIST is a dataset, VGG is a software model (network architecture). You can train/test VGG on MNIST, train/test this spiking network model on MNIST, and compare the accuracy between the two. If you want to compare energy efficiency, you look at the hardware these models are running on (CPU, GPU, FPGA, ASIC), and see how much energy is spent by each of these chips to reach a certain accuracy (preferably on ImageNet). You can also compare the speed of processing (significantly affected by image size): 1. How fast can you get to a certain accuracy? 2. How many images per second can you classify with that accuracy?
    - 0xbear 9y ago
      >> MNIST is a dataset, VGG is a software model Strictly speaking, true. But one discovers early enough that you can screw up a number of things when training a model on something as simple as MNIST and the model will still work, because there's so much redundancy, the correct layers will just train their way around the ones that don't really work. VGG trained for general purpose image classification with a decent sized input is a much better indicator of whether or not some ML hardware improvement is useful.
      - p1esk 9y ago
        You're contradicting yourself. If you're afraid that redundancy will hide hardware problems, then VGG is a bad choice, because it's pretty much the most redundant NN model ever :)
- mtgx 9y ago
  Take that with a huge "3D Xpoint is 1,000x faster than flash" grain of salt.
- chriskanan 9y ago
  Results on MNIST can be highly deceiving because it is so easy. ImageNet would be a good benchmark. Also, are they only comparing against other spiking neural nets instead of deep nets? If so, that doesn't tell much about whether these chips are useful for modern AI. I'd have to see their paper.
- Someone 9y ago
  A million times faster and a thousand times more energy efficient means a thousand times the power usage. I find that hard to believe, so those “other typical spiking neural nets” must be quite different from “typical training systems” => as others said, this is marketing speak.
CoffeeDregs 9y ago
"It’s a future where first responders using image-recognition applications can analyze streetlight camera images and quickly solve missing or abducted person reports." Erp. That's also known as "a massive surveillance network", right? "It’s a future where [your government] using image-recognition applications can analyze streetlight camera images and quickly solve [whether you're involved in pre-crime]." This stuff is going to happen (or is already here) so perhaps it's time to switch to managing pervasive surveillance rather than preventing it.
lend000 9y ago
Looking forward to when there is more information than just a press release (the press release for Intel's 3D X-Point memory was over two years ago, and we still aren't seeing those in production). Regardless, very exciting work, and I look forward to getting my hands on the API/instruction set/manual.
- profquail 9y ago
  3D XPoint has been available for several months now, just not in the DIMM form factor. E.g. the Intel P4800X: https://www.anandtech.com/show/11209/intel-optane-ssd-dc-p4800x-review-a-deep-dive-into-3d-xpoint-enterprise-performance https://www.anandtech.com/show/11209/intel-optane-ssd-dc-p48...
tehsauce 9y ago
This hardware is designed for running spiking neural network models? What is the current state of the art in this field? I was under the impression that training a spiking neural network was somewhat of an unsolved problem, because backprop doesn't easily apply. Anyone have information on this?
- aurelian15 9y ago
  There are several frameworks for the construction of spiking neural networks given an abstract mathematical description of the desired function. The most complete (note that I'm currently a student in the lab that developed this method) is the Neural Engineering Framework [1]. There is a Python reference implementation called Nengo [2] which can target several backends, including neuromorphic hardware. Training deep spiking neural networks is possible as well [3]. [1] http://compneuro.uwaterloo.ca/research/nef.html http://compneuro.uwaterloo.ca/research/nef.html [2] https://www.nengo.ai/ https://www.nengo.ai/ [3] https://arxiv.org/abs/1611.05141 https://arxiv.org/abs/1611.05141
  - p1esk 9y ago
    There's this common claim that spiking networks are somehow more power efficient. Yet I have never seen any evidence to support it, or even any decent explanation why would it ever be the case.
    - aurelian15 9y ago
      They are more power efficient when implemented as analogue circuits. Two main reasons: First, you only need a handful of transistors to implement a single spiking neuron, whereas digital circuits require ten to hundred thousands of transistors for corresponding numerical computations. Second, spiking neural networks are intrinsically asynchronous. All the computing elements (neurons) are independent and only communicate via rare one-bit events (spikes). Correspondingly you (if any) only need a clock for the event bus but not for the vast majority of the chip. See http://web.stanford.edu/group/brainsinsilicon/documents/IEEE2017.pdf http://web.stanford.edu/group/brainsinsilicon/documents/IEEE... for a high-level overview. Edit: To clarify: the asynchronicity argument is valid for digital implementations of spiking neural networks as well, especially as you scale up to networks of a million (or more) neurons. See http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ for an implementation.
bhouston 9y ago
How does this compare with Google's proprietary neural network optimized chip? That seems like a place that Intel could kick ass in and sell tons to Amazon and Microsoft's cloud services for NN acceleration, especially if they supported the main NN libraries. And if Intel isn't pursuing the opportunity to compete for deep learning NN against Google's chips, why aren't they? It seems like a well defined opportunity, more so than whatever these chips are.
- dnautics 9y ago
  I don't know for sure, but google's chips are advanced matrix multiplying engines. the "neuromorphic" chips are (in a very stupid-fied explanation) kind of like imagine a configurable fpgas with more-than-one-bit lines connecting adjacent tiles.
  - copperx 9y ago
    > google's chips are advanced matrix multiplying engines Excuse my ignorance, but aren't GPUs exactly that?
    - rileyphone 9y ago
      From what I can tell, Google's TPUs are boiling GPUs down to the essence of what they do in TensorFlow, namely a "higher volume of reduced precision computation" [1]. [1] https://en.wikipedia.org/wiki/Tensor_processing_unit https://en.wikipedia.org/wiki/Tensor_processing_unit
    - hedgehog 9y ago
      GPUs are generally much more flexible and can run something resembling a normal C program. From the little public information available chips like Google TPU or NVIDIA's Tensor Core (a functional unit on the Volta architecture) are essentially hard-wired to do exactly low precision matrix multiplication. Less flexible but more efficient when you know that's what your workload is. https://devblogs.nvidia.com/parallelforall/inside-volta/ https://devblogs.nvidia.com/parallelforall/inside-volta/ https://www.nextplatform.com/2017/04/05/first-depth-look-googles-tpu-architecture/ https://www.nextplatform.com/2017/04/05/first-depth-look-goo...
visarga 9y ago
> This extremely energy efficient chip, which uses the data to learn and make inferences, gets smarter over time and does not need to be trained in the traditional way. Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. If this was a news site, I'd understand - the same writers cover too many topics, they can't be experts in all. But when it comes straight from Intel, it's weird to see such mistakes from someone who signs as Dr. Michael Mayberry. > Researchers have demonstrated learning at a rate that is a 1 million times improvement compared with other typical spiking neural nets as measured by total operations to achieve a given accuracy when solving MNIST digit recognition problems. MNIST is a joke already, why don't they show off the ImageNet numbers? And spiking neural nets are less efficient compared to normal neural nets, so the 1 million speedup might not be as impressive as it appears at first hand. They don't compare with GPU run TensorFlow or CNTK here, but with other spiking neural nets which are seldom used, mostly as a proof of concept. > A total of 130,000 neurons and 130 million synapses. So the network size is medium - you can do a lot with 130 mil weights, but there are models on GPU that go up to 1 billion weights. Even the "cat detector" unsupervised neural net created by Google in 2011, had 1 billion weights, and that's ages ago in AI time.
- shawnee_ 9y ago
  > Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. Nope, "trained" is correct. The current state of deep learning involves "training" and "inference" as distinct components where entirely different kinds of math are applied to each. GPUs can do training as training is kind of "dumb" and repetitive. CPUs, however, can do training and inference (and also can do allocations among resources dedicated to training and inference). Sure, GPUs are the trendy thing being pumped by stock market day traders these days, but they are only about as efficient as the black Model T Ford in the "traditional" sense.
- Seanny123 9y ago
  > Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. There are two ways to build spiking neural net architectures without training. The [Neural Engineering Framework](https://medium.com/@seanaubin/a-way-around-the-coming-performance-walls-neuromorphic-hardware-with-spiking-neurons-facd4291b201 https://medium.com/@seanaubin/a-way-around-the-coming-perfor...) and [Deneve's approach which I'm more uncertain about](https://forum.nengo.ai/t/robustness-of-deneves-networks/320?u=seanny123 https://forum.nengo.ai/t/robustness-of-deneves-networks/320?...). To be clear, when I say "training" in this context, I mean they don't require iterative rounds of stochastic gradient descent. They just need to be given the function that you want to approximate and then they're done.
m3kw9 9y ago
So this thing only can be trained with a the built in algorithm?
- visarga 9y ago
  Maybe there is a way to port regular neural nets onto spiking nets - you'd train a regular network first using your algorithm of choice, then transfer-learn to the spiking net.
Paradoxinabox 9y ago
Skynet? Skynet.
- kensai 9y ago
  Or Borg?! http://tinyurl.com/borg29 http://tinyurl.com/borg29
caqlar 9y ago
Intel's countermove for Apple's Bionic chip. They are investing more and more in human/environment identification.

135 points by 40acres 9y ago

>"Researchers have demonstrated learning at a rate that is a 1 million times improvement compared with other typical spiking neural nets as measured by total operations to achieve a given accuracy when solving MNIST digit recognition problems." >"Further, it is up to 1,000 times more energy-efficient than general purpose computing required for typical training systems." Wow. That is tremendous.

zitterbewegung 9y ago
Comparing a system to MNIST isn't really an indicator that it provides an actual improvement. Comparing it to VGG would be a better indicator.
- p1esk 9y ago
  MNIST is a dataset, VGG is a software model (network architecture). You can train/test VGG on MNIST, train/test this spiking network model on MNIST, and compare the accuracy between the two. If you want to compare energy efficiency, you look at the hardware these models are running on (CPU, GPU, FPGA, ASIC), and see how much energy is spent by each of these chips to reach a certain accuracy (preferably on ImageNet). You can also compare the speed of processing (significantly affected by image size): 1. How fast can you get to a certain accuracy? 2. How many images per second can you classify with that accuracy?
  - 0xbear 9y ago
    >> MNIST is a dataset, VGG is a software model Strictly speaking, true. But one discovers early enough that you can screw up a number of things when training a model on something as simple as MNIST and the model will still work, because there's so much redundancy, the correct layers will just train their way around the ones that don't really work. VGG trained for general purpose image classification with a decent sized input is a much better indicator of whether or not some ML hardware improvement is useful.
    - p1esk 9y ago
      You're contradicting yourself. If you're afraid that redundancy will hide hardware problems, then VGG is a bad choice, because it's pretty much the most redundant NN model ever :)
mtgx 9y ago
Take that with a huge "3D Xpoint is 1,000x faster than flash" grain of salt.
chriskanan 9y ago
Results on MNIST can be highly deceiving because it is so easy. ImageNet would be a good benchmark. Also, are they only comparing against other spiking neural nets instead of deep nets? If so, that doesn't tell much about whether these chips are useful for modern AI. I'd have to see their paper.
Someone 9y ago
A million times faster and a thousand times more energy efficient means a thousand times the power usage. I find that hard to believe, so those “other typical spiking neural nets” must be quite different from “typical training systems” => as others said, this is marketing speak.

CoffeeDregs 9y ago

"It’s a future where first responders using image-recognition applications can analyze streetlight camera images and quickly solve missing or abducted person reports." Erp. That's also known as "a massive surveillance network", right? "It’s a future where [your government] using image-recognition applications can analyze streetlight camera images and quickly solve [whether you're involved in pre-crime]." This stuff is going to happen (or is already here) so perhaps it's time to switch to managing pervasive surveillance rather than preventing it.

lend000 9y ago

Looking forward to when there is more information than just a press release (the press release for Intel's 3D X-Point memory was over two years ago, and we still aren't seeing those in production). Regardless, very exciting work, and I look forward to getting my hands on the API/instruction set/manual.

profquail 9y ago
3D XPoint has been available for several months now, just not in the DIMM form factor. E.g. the Intel P4800X: https://www.anandtech.com/show/11209/intel-optane-ssd-dc-p4800x-review-a-deep-dive-into-3d-xpoint-enterprise-performance https://www.anandtech.com/show/11209/intel-optane-ssd-dc-p48...

tehsauce 9y ago

This hardware is designed for running spiking neural network models? What is the current state of the art in this field? I was under the impression that training a spiking neural network was somewhat of an unsolved problem, because backprop doesn't easily apply. Anyone have information on this?

aurelian15 9y ago
There are several frameworks for the construction of spiking neural networks given an abstract mathematical description of the desired function. The most complete (note that I'm currently a student in the lab that developed this method) is the Neural Engineering Framework [1]. There is a Python reference implementation called Nengo [2] which can target several backends, including neuromorphic hardware. Training deep spiking neural networks is possible as well [3]. [1] http://compneuro.uwaterloo.ca/research/nef.html http://compneuro.uwaterloo.ca/research/nef.html [2] https://www.nengo.ai/ https://www.nengo.ai/ [3] https://arxiv.org/abs/1611.05141 https://arxiv.org/abs/1611.05141
- p1esk 9y ago
  There's this common claim that spiking networks are somehow more power efficient. Yet I have never seen any evidence to support it, or even any decent explanation why would it ever be the case.
  - aurelian15 9y ago
    They are more power efficient when implemented as analogue circuits. Two main reasons: First, you only need a handful of transistors to implement a single spiking neuron, whereas digital circuits require ten to hundred thousands of transistors for corresponding numerical computations. Second, spiking neural networks are intrinsically asynchronous. All the computing elements (neurons) are independent and only communicate via rare one-bit events (spikes). Correspondingly you (if any) only need a clock for the event bus but not for the vast majority of the chip. See http://web.stanford.edu/group/brainsinsilicon/documents/IEEE2017.pdf http://web.stanford.edu/group/brainsinsilicon/documents/IEEE... for a high-level overview. Edit: To clarify: the asynchronicity argument is valid for digital implementations of spiking neural networks as well, especially as you scale up to networks of a million (or more) neurons. See http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ for an implementation.

bhouston 9y ago

How does this compare with Google's proprietary neural network optimized chip? That seems like a place that Intel could kick ass in and sell tons to Amazon and Microsoft's cloud services for NN acceleration, especially if they supported the main NN libraries. And if Intel isn't pursuing the opportunity to compete for deep learning NN against Google's chips, why aren't they? It seems like a well defined opportunity, more so than whatever these chips are.

dnautics 9y ago
I don't know for sure, but google's chips are advanced matrix multiplying engines. the "neuromorphic" chips are (in a very stupid-fied explanation) kind of like imagine a configurable fpgas with more-than-one-bit lines connecting adjacent tiles.
- copperx 9y ago
  > google's chips are advanced matrix multiplying engines Excuse my ignorance, but aren't GPUs exactly that?
  - rileyphone 9y ago
    From what I can tell, Google's TPUs are boiling GPUs down to the essence of what they do in TensorFlow, namely a "higher volume of reduced precision computation" [1]. [1] https://en.wikipedia.org/wiki/Tensor_processing_unit https://en.wikipedia.org/wiki/Tensor_processing_unit
  - hedgehog 9y ago
    GPUs are generally much more flexible and can run something resembling a normal C program. From the little public information available chips like Google TPU or NVIDIA's Tensor Core (a functional unit on the Volta architecture) are essentially hard-wired to do exactly low precision matrix multiplication. Less flexible but more efficient when you know that's what your workload is. https://devblogs.nvidia.com/parallelforall/inside-volta/ https://devblogs.nvidia.com/parallelforall/inside-volta/ https://www.nextplatform.com/2017/04/05/first-depth-look-googles-tpu-architecture/ https://www.nextplatform.com/2017/04/05/first-depth-look-goo...

visarga 9y ago

> This extremely energy efficient chip, which uses the data to learn and make inferences, gets smarter over time and does not need to be trained in the traditional way. Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. If this was a news site, I'd understand - the same writers cover too many topics, they can't be experts in all. But when it comes straight from Intel, it's weird to see such mistakes from someone who signs as Dr. Michael Mayberry. > Researchers have demonstrated learning at a rate that is a 1 million times improvement compared with other typical spiking neural nets as measured by total operations to achieve a given accuracy when solving MNIST digit recognition problems. MNIST is a joke already, why don't they show off the ImageNet numbers? And spiking neural nets are less efficient compared to normal neural nets, so the 1 million speedup might not be as impressive as it appears at first hand. They don't compare with GPU run TensorFlow or CNTK here, but with other spiking neural nets which are seldom used, mostly as a proof of concept. > A total of 130,000 neurons and 130 million synapses. So the network size is medium - you can do a lot with 130 mil weights, but there are models on GPU that go up to 1 billion weights. Even the "cat detector" unsupervised neural net created by Google in 2011, had 1 billion weights, and that's ages ago in AI time.

shawnee_ 9y ago
> Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. Nope, "trained" is correct. The current state of deep learning involves "training" and "inference" as distinct components where entirely different kinds of math are applied to each. GPUs can do training as training is kind of "dumb" and repetitive. CPUs, however, can do training and inference (and also can do allocations among resources dedicated to training and inference). Sure, GPUs are the trendy thing being pumped by stock market day traders these days, but they are only about as efficient as the black Model T Ford in the "traditional" sense.
Seanny123 9y ago
> Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. There are two ways to build spiking neural net architectures without training. The [Neural Engineering Framework](https://medium.com/@seanaubin/a-way-around-the-coming-performance-walls-neuromorphic-hardware-with-spiking-neurons-facd4291b201 https://medium.com/@seanaubin/a-way-around-the-coming-perfor...) and [Deneve's approach which I'm more uncertain about](https://forum.nengo.ai/t/robustness-of-deneves-networks/320?u=seanny123 https://forum.nengo.ai/t/robustness-of-deneves-networks/320?...). To be clear, when I say "training" in this context, I mean they don't require iterative rounds of stochastic gradient descent. They just need to be given the function that you want to approximate and then they're done.

m3kw9 9y ago

So this thing only can be trained with a the built in algorithm?

visarga 9y ago
Maybe there is a way to port regular neural nets onto spiking nets - you'd train a regular network first using your algorithm of choice, then transfer-learn to the spiking net.

Paradoxinabox 9y ago

Skynet? Skynet.

kensai 9y ago
Or Borg?! http://tinyurl.com/borg29 http://tinyurl.com/borg29

caqlar 9y ago

Intel's countermove for Apple's Bionic chip. They are investing more and more in human/environment identification.