6 ms·

Intel unveils largest-ever AI 'neuromorphic computer' mimicking the human brain

41 points by dosinga 2y ago

abeppu 2y ago
I really wish we could talk about new kinds of hardware without the comparison to the human brain. SNN models capture some characteristics of biological neurons ... but those attributes aren't at all specific to humans. Representing the way information is represented during computation (via spike rate-coding or whatever), using far less power, changing the interaction between computation and memory -- all of these can be exciting and interesting without pretending that it's closely related to human brains.
- falcor84 2y ago
  Apologies for the quack philosophy, but isn't it just that we as human observers of the world can't help but identify our reflection in everything we see or do?
  - knodi123 2y ago
    or, conversely, a quack marketing approach: the more you can relate a product to an existing successful product, the more people will think it's a good idea.
    - abeppu 2y ago
      According to this table [1] the 1.15B artificial neurons they're claiming is just at little below the estimate for raccoon dogs and Tanimbar corella / cockatoo. Even if it's a technically impressive system, "inspired by the racoon dog brain" doesn't make for as good a press release. [1] https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
      - falcor84 2y ago
        Well, even then, it's massive progress that we've reached the realm of mammals. Only 2 more orders of magnitude to go.
        nicklecompte 2y ago
        It's not even close to a mammal. It might have 1bn artificial neurons, while a cat has 700m biological neurons. But a) individual mammal neurons are far more computationally complex than any existing artificial neuron, even in SNNs. b) a cat has 10 trillion synapses compared to the 100 billion artificial synapses in Intel's computer. So even ignoring how simple artificial neurons are, mammal brains have far more topological complexity.
        hollerith 2y ago
        But semiconductors operate so much faster than neurons that it is hard to be confident that AI hadware has not already caught up to cat brains.
        nicklecompte 2y ago
        Well, don't underestimate the sheer back-of-the-envelope power of a cat's brain: 700m neurons x 200Hz per neuron is a LOT of compute, like ~10,000 modern CPU cores in terms of ops/second. Pretty impressive when you consider the thermal efficiency.... Regardless, looking at AI power that way is a mistake. There are already a few supercomputers that can perform more operations/second than the human brain and I wouldn't be too surprised if this neuromorphic computer can perform more ops/s than a cat. But the structure of the brain is important: brains aren't just simply glued-together layers of neurons. The incredible topological complexity and integration of mammal brains surely means that certain "parallelizable algorithms" can be run much more efficiently, and that disparate information can be pulled together more readily for identifying patterns and making mental models. All mammals (and birds) are faster learners than any artificial neural network, and they understand ideas like object permanence and causality, which no AI is even close to getting right. It is reasonable to suppose that the structural complexity of mammal/bird brains is an important part why their cognitive abilities are so much greater than 2024 AI. And I really can't overemphasize how complex mammal neurons are compared to artificial ones: https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... It might not matter that individual biological neurons are slow when by themselves they're as sophisticated as deep artificial neural networks. A neuron might operate at 200Hz but it's not solving "what's the sum of these floating-point vectors?" like a CPU/GPU. It's solving something way more sophisticated: there was a paper recently arguing that humans had individual neurons for visually classifying 2 things, 3 things, etc. (Cats can do this "subitizing" almost as well as we can and probably have their own neurons for it, but their understanding of numbers seems shallower than a primate's.) So: - the artificial neurons are far simpler than cat neurons - the links between the artificial neurons in neuromorphic computers are far simpler than the links between cat neurons in cat brains - even in naive ops/second cats are not slouches Simply matching the number of cat neurons and waving hands about "fast semiconductors" is not enough for me to think this machine is as powerful as a cat's brain. (And even if Intel made a computer theoretically capable of emulating a cat, I suspect it'll take several decades to develop AI software which is sophisticated enough to do the job.)
- phkahler 2y ago
  >> all of these can be exciting and interesting without pretending that it's closely related to human brains Why do you say they are not closely related? They are certainly an analogy to parts of the human brain. They are inspired by the way real neurons work at some level. But using the word "pretending" you sound as if you think these similarities are pure fiction. Why is that?
  - nicklecompte 2y ago
    I have three basic problems with using "brain" and especially "human brain": 1) Nobody is even bothering to explain why this is like a human brain rather than 1M spider brains or 1B nematode brains - I suspect 1M spider brains is probably closer to reality, and I doubt that adds up to one human. The choice of "human" is a marketing gimmick. 2) Even in nematode brains, each neuron is very different, in a way that can't be explained by a small array of floating-point numbers. These SNN architectures are too homogenous and simple to explain the actual behavior of individual neurons. (See https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... which discusses standard ANNs, but I strongly doubt a single layer of SNN neurons can actually emulate an individual rat neuron.) 3) Epigenetic effects are clearly important for thinking and memory formation, yet totally ignored by these architectures: https://www.smithsonianmag.com/smart-news/scientists-say-they-have-transferred-memories-between-snails-180969087/ https://www.smithsonianmag.com/smart-news/scientists-say-the...
  - deleted 2y ago
    [deleted]
  - Retric 2y ago
    Because it’s nowhere close to the scale of the human brain, calling it a mouse brain would be closer. People have 100x the neurons and 10,000x the synaptic connections and wildly different architecture, so it’s really got nothing in more in common with a human brain over some other kind of brain.
- Solvency 2y ago
  this will never happen. why? because humans are outrageously self-centeredly narcissistic. so narcissistic that we invented religions where god is just an omnipotent/omnipowerful version of a man, and then we go "god created man in his image" to justify it. humans can't help but be obsessed with other humans, the same way every species is hardwired in a specific way to relate to their own species. the idea of any computer system "brain" not being compared and contrasted to our own is against our very narcissistic nature.
- BobaFloutist 2y ago
  I do think there's some value in recognizing when people create something directly inspired by looking at how the brain works and trying to mimic it with hardware. Whether or not that's what's happening here is another matter.
  - abeppu 2y ago
    ... but the thing being mimicked is just as true of a mouse brain or (I think?) any vertebrate brain.
    - deleted 2y ago
      [deleted]
    - deleted 2y ago
      [deleted]
    - BobaFloutist 2y ago
      Then the article title should probably be "mimicking biological brains"! Oh well.
- 2OEH8eoCRo0 2y ago
  Remove the human brain part and try to explain why this is news to a layperson.
  - abeppu 2y ago
    I'm not an expert in these systems, and their announcement materials don't make a bunch of stuff clear, but my attempt at summarizing for a layperson would be: - Current computers represent values as patterns of binary values. Analog computers sometimes represent values as e.g. voltages within a range. SNNs represent values with the timing of "spike" events. You could think of this like if you and your friends arranged a scheme to communicate using only the original Yo app, just by varying timing and frequency. - Current computers move values back and forth between memory and the CPU, and has very little room to hold data "close" to the CPU. Suppose your e-reader can only have one doc page open at a time; if you need to figure something out by closely comparing values between several docs, you might spend more time opening/closing/finding/paging through docs than actually reading them or thinking. These systems put (some) memory to be held by the same physical components which are doing the computations, so they aren't moving stuff back and forth in the same way. - However, current computers are quiet general in what they can do. In exchange for speed an efficiency, these systems are not general purpose. For example, b/c they have 1.15B "neurons" and 128B "synapses", any value can only (directly) influence on average 112 other values. You could think of this a bit like if Excel only let your formula reference a capped number of cells. - But for research problems which fit these constraints, this system can be extremely fast and use less power than a normal computer would.
    - 2OEH8eoCRo0 2y ago
      If I'm LARPing as a layperson then you've lost me in the first sentence.
- AndrewKemendo 2y ago
  Human capacity is the benchmark for capacity comparison for every thing humans care about on the planet For example even though the impulse-striking power or multi spectral vision system of the mantis shrimp is much more capable than human systems, it is not a benchmark for any of our systems Therefore we’re going to continue to evaluate everything. On human action until we create something that surpasses it. Comparing it to the brain, I agree is silly comparing them to human capabilities however, (which is effectively what people are saying when they compared to the brain, even though they don’t know that) is very reasonable
cwillu 2y ago
“In neuromorphic computing, however, a "spike input" — a set of discrete electrical signals — is fed into the spiking neural networks (SNNs), represented by the processors. Where software-based neural networks are a collection of machine learning algorithms arranged to mimic the human brain, SNNs are a physical embodiment of how that information is transmitted. It allows for parallel processing and spike outputs are measured following calculations. Like the brain, Hala Point and the Loihi 2 processors use these SNNs, where different nodes are connected and information is processed at different layers, similar to neurons in the brain.” My impression is that there hadn't been much success with SNN's yet.
- andai 2y ago
  You are one though.
  - umvi 2y ago
    It's a bit cargo cult-y to claim that synthetic SNNs and human brains are equivalent just because they share some similarities.
    - andai 2y ago
      Isn't a human brain a spiking neural network? And a successful one at that? I didn't say what we can build now is identical to a human brain (...why would anyone say that?), I said we have proof that the concept is viable, because a spiking neural network is literally the thing reading these words right now.
  - cwillu 2y ago
    Having checked a mirror, I stand by my initial assessment.
    - andai 2y ago
      My point was that humans are arguably quite successful, their success is due to their neural networks, and their neural networks are spiking (in contrast to the ones used in LLMs). The spiking might be important, or it might not. (One way to find out!) But we certainly have proof that it can be made to work!
p1esk 2y ago
Last time I checked SNNs still don’t work as well as traditional NNs. Why are they working on hardware before getting software to work?
- johnnyjeans 2y ago
  Traditional NNs didn't work very well either, until we were able to run them at massive scales. That's why they didn't see much adoption or use until the late 2000s/early 2010s. SNNs are much more difficult to run at scale, but that difficulty is solved with analogue computers[1][2]. We don't have details, but I'd bet this is just Intel's crack at a modern analogue processor, with some design concessions specifically for running SNNs. [1] https://www.mdpi.com/2079-9268/13/4/55 https://www.mdpi.com/2079-9268/13/4/55 [2] https://www.mdpi.com/1424-8220/23/14/6275 https://www.mdpi.com/1424-8220/23/14/6275
  - p1esk 2y ago
    Loihi is a digital chip.
- orbital-decay 2y ago
  SNNs are behind, but not that far behind, especially after recent astrocyte-related advancements, and if implemented correctly in hardware they consume a fraction of power. SNNs don't get nearly the same amount of attention as ordinary networks largely because they lack efficient hardware to run on (just like ordinary NNs until the advent of GPGPU). It's not exactly new hardware though, it's just a cluster of 1152 existing Intel Loihi 2 neuromorphic chips.
  - p1esk 2y ago
    Which “recent astrocyte-related advancements”?
deleted 2y ago
[deleted]

41 points by dosinga 2y ago

I really wish we could talk about new kinds of hardware without the comparison to the human brain. SNN models capture some characteristics of biological neurons ... but those attributes aren't at all specific to humans. Representing the way information is represented during computation (via spike rate-coding or whatever), using far less power, changing the interaction between computation and memory -- all of these can be exciting and interesting without pretending that it's closely related to human brains.

falcor84 2y ago
Apologies for the quack philosophy, but isn't it just that we as human observers of the world can't help but identify our reflection in everything we see or do?
- knodi123 2y ago
  or, conversely, a quack marketing approach: the more you can relate a product to an existing successful product, the more people will think it's a good idea.
  - abeppu 2y ago
    According to this table [1] the 1.15B artificial neurons they're claiming is just at little below the estimate for raccoon dogs and Tanimbar corella / cockatoo. Even if it's a technically impressive system, "inspired by the racoon dog brain" doesn't make for as good a press release. [1] https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
    - falcor84 2y ago
      Well, even then, it's massive progress that we've reached the realm of mammals. Only 2 more orders of magnitude to go.
      - nicklecompte 2y ago
        It's not even close to a mammal. It might have 1bn artificial neurons, while a cat has 700m biological neurons. But a) individual mammal neurons are far more computationally complex than any existing artificial neuron, even in SNNs. b) a cat has 10 trillion synapses compared to the 100 billion artificial synapses in Intel's computer. So even ignoring how simple artificial neurons are, mammal brains have far more topological complexity.
        hollerith 2y ago
        But semiconductors operate so much faster than neurons that it is hard to be confident that AI hadware has not already caught up to cat brains.
        nicklecompte 2y ago
        Well, don't underestimate the sheer back-of-the-envelope power of a cat's brain: 700m neurons x 200Hz per neuron is a LOT of compute, like ~10,000 modern CPU cores in terms of ops/second. Pretty impressive when you consider the thermal efficiency.... Regardless, looking at AI power that way is a mistake. There are already a few supercomputers that can perform more operations/second than the human brain and I wouldn't be too surprised if this neuromorphic computer can perform more ops/s than a cat. But the structure of the brain is important: brains aren't just simply glued-together layers of neurons. The incredible topological complexity and integration of mammal brains surely means that certain "parallelizable algorithms" can be run much more efficiently, and that disparate information can be pulled together more readily for identifying patterns and making mental models. All mammals (and birds) are faster learners than any artificial neural network, and they understand ideas like object permanence and causality, which no AI is even close to getting right. It is reasonable to suppose that the structural complexity of mammal/bird brains is an important part why their cognitive abilities are so much greater than 2024 AI. And I really can't overemphasize how complex mammal neurons are compared to artificial ones: https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... It might not matter that individual biological neurons are slow when by themselves they're as sophisticated as deep artificial neural networks. A neuron might operate at 200Hz but it's not solving "what's the sum of these floating-point vectors?" like a CPU/GPU. It's solving something way more sophisticated: there was a paper recently arguing that humans had individual neurons for visually classifying 2 things, 3 things, etc. (Cats can do this "subitizing" almost as well as we can and probably have their own neurons for it, but their understanding of numbers seems shallower than a primate's.) So: - the artificial neurons are far simpler than cat neurons - the links between the artificial neurons in neuromorphic computers are far simpler than the links between cat neurons in cat brains - even in naive ops/second cats are not slouches Simply matching the number of cat neurons and waving hands about "fast semiconductors" is not enough for me to think this machine is as powerful as a cat's brain. (And even if Intel made a computer theoretically capable of emulating a cat, I suspect it'll take several decades to develop AI software which is sophisticated enough to do the job.)
phkahler 2y ago
>> all of these can be exciting and interesting without pretending that it's closely related to human brains Why do you say they are not closely related? They are certainly an analogy to parts of the human brain. They are inspired by the way real neurons work at some level. But using the word "pretending" you sound as if you think these similarities are pure fiction. Why is that?
- nicklecompte 2y ago
  I have three basic problems with using "brain" and especially "human brain": 1) Nobody is even bothering to explain why this is like a human brain rather than 1M spider brains or 1B nematode brains - I suspect 1M spider brains is probably closer to reality, and I doubt that adds up to one human. The choice of "human" is a marketing gimmick. 2) Even in nematode brains, each neuron is very different, in a way that can't be explained by a small array of floating-point numbers. These SNN architectures are too homogenous and simple to explain the actual behavior of individual neurons. (See https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... which discusses standard ANNs, but I strongly doubt a single layer of SNN neurons can actually emulate an individual rat neuron.) 3) Epigenetic effects are clearly important for thinking and memory formation, yet totally ignored by these architectures: https://www.smithsonianmag.com/smart-news/scientists-say-they-have-transferred-memories-between-snails-180969087/ https://www.smithsonianmag.com/smart-news/scientists-say-the...
- deleted 2y ago
  [deleted]
- Retric 2y ago
  Because it’s nowhere close to the scale of the human brain, calling it a mouse brain would be closer. People have 100x the neurons and 10,000x the synaptic connections and wildly different architecture, so it’s really got nothing in more in common with a human brain over some other kind of brain.
Solvency 2y ago
this will never happen. why? because humans are outrageously self-centeredly narcissistic. so narcissistic that we invented religions where god is just an omnipotent/omnipowerful version of a man, and then we go "god created man in his image" to justify it. humans can't help but be obsessed with other humans, the same way every species is hardwired in a specific way to relate to their own species. the idea of any computer system "brain" not being compared and contrasted to our own is against our very narcissistic nature.
BobaFloutist 2y ago
I do think there's some value in recognizing when people create something directly inspired by looking at how the brain works and trying to mimic it with hardware. Whether or not that's what's happening here is another matter.
- abeppu 2y ago
  ... but the thing being mimicked is just as true of a mouse brain or (I think?) any vertebrate brain.
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - BobaFloutist 2y ago
    Then the article title should probably be "mimicking biological brains"! Oh well.
2OEH8eoCRo0 2y ago
Remove the human brain part and try to explain why this is news to a layperson.
- abeppu 2y ago
  I'm not an expert in these systems, and their announcement materials don't make a bunch of stuff clear, but my attempt at summarizing for a layperson would be: - Current computers represent values as patterns of binary values. Analog computers sometimes represent values as e.g. voltages within a range. SNNs represent values with the timing of "spike" events. You could think of this like if you and your friends arranged a scheme to communicate using only the original Yo app, just by varying timing and frequency. - Current computers move values back and forth between memory and the CPU, and has very little room to hold data "close" to the CPU. Suppose your e-reader can only have one doc page open at a time; if you need to figure something out by closely comparing values between several docs, you might spend more time opening/closing/finding/paging through docs than actually reading them or thinking. These systems put (some) memory to be held by the same physical components which are doing the computations, so they aren't moving stuff back and forth in the same way. - However, current computers are quiet general in what they can do. In exchange for speed an efficiency, these systems are not general purpose. For example, b/c they have 1.15B "neurons" and 128B "synapses", any value can only (directly) influence on average 112 other values. You could think of this a bit like if Excel only let your formula reference a capped number of cells. - But for research problems which fit these constraints, this system can be extremely fast and use less power than a normal computer would.
  - 2OEH8eoCRo0 2y ago
    If I'm LARPing as a layperson then you've lost me in the first sentence.
AndrewKemendo 2y ago
Human capacity is the benchmark for capacity comparison for every thing humans care about on the planet For example even though the impulse-striking power or multi spectral vision system of the mantis shrimp is much more capable than human systems, it is not a benchmark for any of our systems Therefore we’re going to continue to evaluate everything. On human action until we create something that surpasses it. Comparing it to the brain, I agree is silly comparing them to human capabilities however, (which is effectively what people are saying when they compared to the brain, even though they don’t know that) is very reasonable

cwillu 2y ago

“In neuromorphic computing, however, a "spike input" — a set of discrete electrical signals — is fed into the spiking neural networks (SNNs), represented by the processors. Where software-based neural networks are a collection of machine learning algorithms arranged to mimic the human brain, SNNs are a physical embodiment of how that information is transmitted. It allows for parallel processing and spike outputs are measured following calculations. Like the brain, Hala Point and the Loihi 2 processors use these SNNs, where different nodes are connected and information is processed at different layers, similar to neurons in the brain.” My impression is that there hadn't been much success with SNN's yet.

andai 2y ago
You are one though.
- umvi 2y ago
  It's a bit cargo cult-y to claim that synthetic SNNs and human brains are equivalent just because they share some similarities.
  - andai 2y ago
    Isn't a human brain a spiking neural network? And a successful one at that? I didn't say what we can build now is identical to a human brain (...why would anyone say that?), I said we have proof that the concept is viable, because a spiking neural network is literally the thing reading these words right now.
- cwillu 2y ago
  Having checked a mirror, I stand by my initial assessment.
  - andai 2y ago
    My point was that humans are arguably quite successful, their success is due to their neural networks, and their neural networks are spiking (in contrast to the ones used in LLMs). The spiking might be important, or it might not. (One way to find out!) But we certainly have proof that it can be made to work!

p1esk 2y ago

Last time I checked SNNs still don’t work as well as traditional NNs. Why are they working on hardware before getting software to work?

johnnyjeans 2y ago
Traditional NNs didn't work very well either, until we were able to run them at massive scales. That's why they didn't see much adoption or use until the late 2000s/early 2010s. SNNs are much more difficult to run at scale, but that difficulty is solved with analogue computers[1][2]. We don't have details, but I'd bet this is just Intel's crack at a modern analogue processor, with some design concessions specifically for running SNNs. [1] https://www.mdpi.com/2079-9268/13/4/55 https://www.mdpi.com/2079-9268/13/4/55 [2] https://www.mdpi.com/1424-8220/23/14/6275 https://www.mdpi.com/1424-8220/23/14/6275
- p1esk 2y ago
  Loihi is a digital chip.
orbital-decay 2y ago
SNNs are behind, but not that far behind, especially after recent astrocyte-related advancements, and if implemented correctly in hardware they consume a fraction of power. SNNs don't get nearly the same amount of attention as ordinary networks largely because they lack efficient hardware to run on (just like ordinary NNs until the advent of GPGPU). It's not exactly new hardware though, it's just a cluster of 1152 existing Intel Loihi 2 neuromorphic chips.
- p1esk 2y ago
  Which “recent astrocyte-related advancements”?

deleted 2y ago

[deleted]