4 ms·

I believe the argument is that you can also encode information in the time domain. If we just look at spikes as a different numerical representation, then they

I believe the argument is that you can also encode information in the time domain. If we just look at spikes as a different numerical representation, then they are clearly inferior. For example, consider that encoding the number 7 will require seven consecutive pulses on a single spiking line. Encoding the number in binary will require one pulse on three parallel lines. Binary encoding wins 7x in speed and 7/3=2.333x in power efficiency... On the other hand, if we assume that we are able to encode information in the gaps between pulses, then things quickly change.

dist-epoch 1y ago
> you can also encode information in the time domain. Also known as a serial interface. They are very successful: PCIe lane, SATA, USB.
- cpldcpu 1y ago
  These interfaces use serialized binary encoding. SNNs are more similar to pulse density modulation (PDM), if you are looking for an electronic equivalent.
CuriouslyC 1y ago
https://en.wikipedia.org/wiki/Frequency-division_multiplexing https://en.wikipedia.org/wiki/Frequency-division_multiplexin... The brain is doing shit like this.
- drob518 1y ago
  And more, I suspect.
- deleted 1y ago
  [deleted]
HarHarVeryFunny 1y ago
I think the main benefit of a neuromorphic design would be to make it dataflow driven (asynchronous event driven - don't update neuron outputs unless their inputs change) rather than synchronous, which is the big power efficiency unlock. This doesn't need to imply a spiking design though - that seems more of an implementation detail, at least as far as dataflow goes. Nature seems to use spike firing rates to encode activation strength. In the brain the relative timing/ordering of different neurons asynchronously activating (A before B, or B before A) is also used (spike-timing-dependent plasticity - STDP) as a learning signal to strengthen or weaken connection strengths, presumably to learn sequence prediction in this asynchronous environment. STDP also doesn't imply that spikes or single neuron spike train inter-spike timings are necessary - an activation event with a strength and timestamp would seem to be enough to implement a digital dataflow design, although ultimately a custom analog design may be more efficient.
- GregarianChild 1y ago
  Can you explain the benefit of renaming dataflow as 'neuromorphic'? You do understand that dataflow architectures have been tried many many times? See [1] for a brief history. MIT had a bit dataflow lab for many years (lead by the recently deceased Arvind). What is the benefit of re-inventing dataflow architectures by complete amateurs who are not at all aware of the 1/2 century research tradition on dataflow architecture, and the very clear and concrete reasons when this architecture has so far failed whenever it was tried for general purpose processors? We can not even apply Santayana's "those who forget their history are condemned to repeat it because the 'neuromorphic' milieu doesn't even bother understanding this history. [1] https://csg.csail.mit.edu/Dataflow/talks/DennisTalk.pdf https://csg.csail.mit.edu/Dataflow/talks/DennisTalk.pdf
  - HarHarVeryFunny 1y ago
    > Can you explain the benefit of renaming dataflow as 'neuromorphic'? Neuromorphic just means brain-like or brain inspired, and brains operate in asynchronous dataflow type fashion. I'm not sure how you read into what I wrote that I was "renaming dataflow as neuromorphic", which was certainly not what I meant. I wonder if you regard Steve Furber (who I knew from Acorn), designer of the ARM CPU, as a "complete amateur"? He also designed the AMULET async processors, as well as for that matter the SpiNNaker system for spiking neural network research. In any case, async (dataflow) processor design, while complex, clearly isn't an impossible task, and at some point in the future when the need arises (mobile robotics?) and there is sufficient financial incentive, I expect we'll see it utilized in commercial systems. I'm not sure why you focus on "general purpose processors" given that we're talking about ANNs and neuromorphic systems. A custom chip would seem a better bet if the goal is to minimize power usage.
    - GregarianChild 1y ago
      I don't rate Furber as a "complete amateur", but he's the exception in this milieu. > Neuromorphic just means brain-like or brain inspired, I don't even see any evidence that 'neuromorphic' architecture is brain inspired in a non-trivial sense. Can you please provide evidence, for example, a non-trivial mapping between 'neuromorphic' architectures (say SpiNNaker) and the SOTA models of the brain that we have, e.g. the existing data-driven model simulating C. elegans brain (the MetaWorm project)? As Steve Furber also says (personal communication): we don't know enough of how the brain works to have computer architectures that can meaningfully inspired by brains. The "neuro-" prefix is marketing. [1] documents this use and dates it back to the 19th century. See also • Neuroergonomics • Neurotypical • Neurodivergent • Neurodiverse • Neurosis • Neuroethics • Neuroeconomics • Neuromarketing • Neurolaw • Neurosecurity • Neuropsychology • Neuropsychoanalysis • Neurotheology • Neuro-Linguistic Programming • Neurogastronomy I have seen all the above used without irony. > brains operate in asynchronous dataflow type fashion. That's a questionable statement. To the best of my knowledge, there is no consensus as of 2025 of how to model even a single neuron. (Function of synapse is even less understood). When I asked Steve Furber what 'neuromorphic meant, he said: "There are many things today described as neuromphric. Mead would not call SpiNNaker as neuromorphic!" Furber also said: "Neuromorphic status: attracts no money, but works (in the sense of accelerate in niche domains)". Upon my asking what niche domains, he said: "brain simulation but nothing else". (He was referring to SpiNNacker). I asked him if SpiNNaker can accelerate back-propagation and he said: "no, because the brain does not do back-propagation". > async (dataflow) processor design, while complex, clearly isn't an impossible task I did not say it was impossible. It has been done many times, see my references to Arvind's lab at MIT (I spent some time there). The problem with async (dataflow) processor design is that it consistently fails to live up to its promises (PPA). There are specific technical reason for that that are quite well understood. > why you focus on "general purpose processors" given that we're talking about ANNs and neuromorphic systems. Because the 'neuromorphic' marketing often reads like they want to build more efficient 'brain-inspired' general purpose computers. Certainly the dataflow architectures (a la Arvind/MIT) tried to. This is one of the many issues with the 'neuromorphic' milieu: they are really vague about their goals. If they would restrict their claims to certain classes of accelerators, then their claims would be less delusional. > the goal is to minimize power usage. If that is the goal, they are also not very successful. On CMOS silicon changes from 0 to 1 or 0 to 1 is what consumes most of the power, this would make the constant spiking expensive, no? [1] K. S. Kendler, A history of metaphorical brain talk in psychiatry. https://www.nature.com/articles/s41380-025-03053-6 https://www.nature.com/articles/s41380-025-03053-6
nickpsecurity 1y ago
"I believe the argument is that you can also encode information in the time domain." Brain research showed that's happening, too. You'll see many models like this if you DuckDuckGo for "spiking" "temporal" "encoding" or subtitute "time" for temporal. You can further use "neural" "network" or "brain" focus it on sub-fields.

dist-epoch 1y ago

> you can also encode information in the time domain. Also known as a serial interface. They are very successful: PCIe lane, SATA, USB.

cpldcpu 1y ago
These interfaces use serialized binary encoding. SNNs are more similar to pulse density modulation (PDM), if you are looking for an electronic equivalent.

CuriouslyC 1y ago

https://en.wikipedia.org/wiki/Frequency-division_multiplexing https://en.wikipedia.org/wiki/Frequency-division_multiplexin... The brain is doing shit like this.

drob518 1y ago
And more, I suspect.
deleted 1y ago
[deleted]

HarHarVeryFunny 1y ago

I think the main benefit of a neuromorphic design would be to make it dataflow driven (asynchronous event driven - don't update neuron outputs unless their inputs change) rather than synchronous, which is the big power efficiency unlock. This doesn't need to imply a spiking design though - that seems more of an implementation detail, at least as far as dataflow goes. Nature seems to use spike firing rates to encode activation strength. In the brain the relative timing/ordering of different neurons asynchronously activating (A before B, or B before A) is also used (spike-timing-dependent plasticity - STDP) as a learning signal to strengthen or weaken connection strengths, presumably to learn sequence prediction in this asynchronous environment. STDP also doesn't imply that spikes or single neuron spike train inter-spike timings are necessary - an activation event with a strength and timestamp would seem to be enough to implement a digital dataflow design, although ultimately a custom analog design may be more efficient.

GregarianChild 1y ago
Can you explain the benefit of renaming dataflow as 'neuromorphic'? You do understand that dataflow architectures have been tried many many times? See [1] for a brief history. MIT had a bit dataflow lab for many years (lead by the recently deceased Arvind). What is the benefit of re-inventing dataflow architectures by complete amateurs who are not at all aware of the 1/2 century research tradition on dataflow architecture, and the very clear and concrete reasons when this architecture has so far failed whenever it was tried for general purpose processors? We can not even apply Santayana's "those who forget their history are condemned to repeat it because the 'neuromorphic' milieu doesn't even bother understanding this history. [1] https://csg.csail.mit.edu/Dataflow/talks/DennisTalk.pdf https://csg.csail.mit.edu/Dataflow/talks/DennisTalk.pdf
- HarHarVeryFunny 1y ago
  > Can you explain the benefit of renaming dataflow as 'neuromorphic'? Neuromorphic just means brain-like or brain inspired, and brains operate in asynchronous dataflow type fashion. I'm not sure how you read into what I wrote that I was "renaming dataflow as neuromorphic", which was certainly not what I meant. I wonder if you regard Steve Furber (who I knew from Acorn), designer of the ARM CPU, as a "complete amateur"? He also designed the AMULET async processors, as well as for that matter the SpiNNaker system for spiking neural network research. In any case, async (dataflow) processor design, while complex, clearly isn't an impossible task, and at some point in the future when the need arises (mobile robotics?) and there is sufficient financial incentive, I expect we'll see it utilized in commercial systems. I'm not sure why you focus on "general purpose processors" given that we're talking about ANNs and neuromorphic systems. A custom chip would seem a better bet if the goal is to minimize power usage.
  - GregarianChild 1y ago
    I don't rate Furber as a "complete amateur", but he's the exception in this milieu. > Neuromorphic just means brain-like or brain inspired, I don't even see any evidence that 'neuromorphic' architecture is brain inspired in a non-trivial sense. Can you please provide evidence, for example, a non-trivial mapping between 'neuromorphic' architectures (say SpiNNaker) and the SOTA models of the brain that we have, e.g. the existing data-driven model simulating C. elegans brain (the MetaWorm project)? As Steve Furber also says (personal communication): we don't know enough of how the brain works to have computer architectures that can meaningfully inspired by brains. The "neuro-" prefix is marketing. [1] documents this use and dates it back to the 19th century. See also • Neuroergonomics • Neurotypical • Neurodivergent • Neurodiverse • Neurosis • Neuroethics • Neuroeconomics • Neuromarketing • Neurolaw • Neurosecurity • Neuropsychology • Neuropsychoanalysis • Neurotheology • Neuro-Linguistic Programming • Neurogastronomy I have seen all the above used without irony. > brains operate in asynchronous dataflow type fashion. That's a questionable statement. To the best of my knowledge, there is no consensus as of 2025 of how to model even a single neuron. (Function of synapse is even less understood). When I asked Steve Furber what 'neuromorphic meant, he said: "There are many things today described as neuromphric. Mead would not call SpiNNaker as neuromorphic!" Furber also said: "Neuromorphic status: attracts no money, but works (in the sense of accelerate in niche domains)". Upon my asking what niche domains, he said: "brain simulation but nothing else". (He was referring to SpiNNacker). I asked him if SpiNNaker can accelerate back-propagation and he said: "no, because the brain does not do back-propagation". > async (dataflow) processor design, while complex, clearly isn't an impossible task I did not say it was impossible. It has been done many times, see my references to Arvind's lab at MIT (I spent some time there). The problem with async (dataflow) processor design is that it consistently fails to live up to its promises (PPA). There are specific technical reason for that that are quite well understood. > why you focus on "general purpose processors" given that we're talking about ANNs and neuromorphic systems. Because the 'neuromorphic' marketing often reads like they want to build more efficient 'brain-inspired' general purpose computers. Certainly the dataflow architectures (a la Arvind/MIT) tried to. This is one of the many issues with the 'neuromorphic' milieu: they are really vague about their goals. If they would restrict their claims to certain classes of accelerators, then their claims would be less delusional. > the goal is to minimize power usage. If that is the goal, they are also not very successful. On CMOS silicon changes from 0 to 1 or 0 to 1 is what consumes most of the power, this would make the constant spiking expensive, no? [1] K. S. Kendler, A history of metaphorical brain talk in psychiatry. https://www.nature.com/articles/s41380-025-03053-6 https://www.nature.com/articles/s41380-025-03053-6

nickpsecurity 1y ago

"I believe the argument is that you can also encode information in the time domain." Brain research showed that's happening, too. You'll see many models like this if you DuckDuckGo for "spiking" "temporal" "encoding" or subtitute "time" for temporal. You can further use "neural" "network" or "brain" focus it on sub-fields.