6 ms·

Nanowire synapses 30,000x faster than nature’s

gardenfelder 4y ago
The piece is chock full of interesting findings, using terms biologists routinely use. But, do those terms they use, e.g. neuron, synapse mean the same thing they do for biologists? For instances, we know that synapses can be one of excitatory or inhibitory, and we know that neurons are bathed in a wash of hormones. Neurons make hormones which serve other functions throughout the brain. For instance "Neuron-Derived Estrogen Regulates Synaptic Plasticity and Memory" [1]. How does the linked work stack up against that? [1] https://www.jneurosci.org/content/39/15/2792 https://www.jneurosci.org/content/39/15/2792
- superkuh 4y ago
  All that, more, and it seems like every computational biology analogy just completely forgets about the most common cell type in the brain: astrocytes. And then there are things like axo-axonal transmission that totally blow up the simple models, https://www.cell.com/neuron/fulltext/S0896-6273(22)00656-0 https://www.cell.com/neuron/fulltext/S0896-6273(22)00656-0
  - ad404b8a372f2b9 4y ago
    Can you elaborate on how the axo-axonal transmissions blow up our simple models? I couldn't understand anything from that summary. Would it be the equivalent of edges communicating between each other in artifical neural networks?
  - bismuthcrystal 4y ago
    Biologists just won't allows us to have any fun. It is always this kind of rhetoric: "what, are you modeling the brain without considering the influence of <insert obscure type of cell> on the hormone regulated blood flow around ion pump circuits during chinese new year neuron firing patterns? you are obviously bounded to fail..." This whole AI field keeps on failing because people like to overthink things. Did Michelangelo need to know molecular chemistry to make sculptures? Why do people pretend there is no artistic component to building AI? Rant finished.
    - bee_rider 4y ago
      I don't think this is the case; the whole field of AI seems pretty healthy at the moment, not failing, and not all that worried about the inaccuracy of their model (It's only a model /Patsy -- but it can still host the whole song-and-dance).
    - IIAOPSW 4y ago
      While I agree there is a large contingent of people who seem to believe every detail of real neurons are necessary to cognition (up to and including the atoms), and while I tend to agree those people are way off the mark about which details are actually relevant to the algorithms of cognition versus which are specific to natures implementation thereof, I also think the existing AI field goes too far in the other direction and oversimplifies more than it can get away with. So far we've been able to gloss over our mistake through the raw brute force of voluminous training data and GPU power. It works in the same way using a hammer to drive a screw into a wall works. Sure, we can do it to an extent, but there's a much better way. We need to figure out how to use a screwdriver. And by screwdriver I mean slightly more sophisticated artificial neuron.
- orbifold 4y ago
  Short answer is no. The field is full with tenuous analogies. Then again „Neural Networks“ are also at best metaphorically related. More accurate existing Neuron models are actually also plagued by lots of limitations among them that they are typically implemented in 3 ancient domain specific languages with lots and lots of hardcoded constants copied from research papers.
  - a-dub 4y ago
    spiking neural networks are interesting though, and new computational substrates that allow for experiments at larger scales could produce some interesting results. today's sum n' squash (sometimes not even squash) graph networks were just kind of a curiosity before gpus turned them into a new very successful computational paradigm. maybe we'll see something similar with these high element count optical spiking graphs, even if they aren't great approximations of the real biology. i like to think that a new analog computational substrate (or mixed analog and digital system) will be what drives the next leap in machine computation.
    - mjhay 4y ago
      I'm excited to see where spiking NNs go. Something like that is needed to progress now that conventional NNs on GPUs are pretty much tapped out (in terms of non-incremental advances) from their power consumption and the end of Moore's law. Things really do need to be more hardware-efficient.
- mdp2021 4y ago
  > But, do those terms they use, e.g. neuron, synapse mean the same thing they do for biologists They are not meant to. This is not "brain simulation" or similar - which exists, but is a different matter. This context is instead about neuromorphic computing, as hardware implementation of components for Artificial Neural Networks. And results seem to be remarkable: > They calculated that the synapses are capable of spike rates exceeding 10 million hertz while consuming roughly 33 attojoules of power per synaptic event (an attojoule is 10-18 of a joule) The comparison with biological neuro-transmission is just indicative - for trivia, for curiosity. -- Edit: on the contrary, these devices aim to be in a way simpler than ANN's neurons (far from aiming to be as complex as cerebral neurons): > By only rarely firing spikes, these devices shuffle around much less data than typical artificial neural networks and, in principle, require much less power and communication bandwidth That is because the underlying aim is to achieve using a single photon for communication, with an immediate potential practical use in ANNs.
  - bee_rider 4y ago
    We should also note that the article only references the over-dramatic comparison to biological neurons in passing in the first paragraph (wonder if it is an "author gets it, but doesn't get to pick the title" type problem).
- dr_dshiv 4y ago
  The main idea is oscillatory neurodynamics. When you have a bunch of tuned oscillators, you can produce incredible computational capacity.
raziel2701 4y ago
These work only at a few Kelvin because they need superconducting nanowires. Will this type of stuff ever leave the lab?
TheRealPomax 4y ago
Not exactly impressive, nature is famous for calling it quits once something works well enough. Anyone who wants to can create a non-biological neuron/axion connection that runs orders of magnitude faster. The real question is "but does it actually do something" because that's where nature shines pretty well.
- miovoid 4y ago
  Turing and Church said yes.
  - TheRealPomax 4y ago
    Said yes to... what? That artificial neurons can do something useful? Because those weren't a thing when they were scienceing it up.
- yourapostasy 4y ago
  > ...nature is famous for calling it quits once something works well enough... Great, so we're just bags of meat sashaying around with loads of technical debt baggage via the slapdash coding delivered by nature in a multi-billion year project. The ongoing result of always picking good and cheap over fast.
  - TheRealPomax 4y ago
    you bet, nature looked at min-max strategies and went "k but what if min-med instead?" then did that for a few billion years.
- TeacherTortoise 4y ago
  > nature is famous for calling it quits Yes, evolution builds upon what went before rather than starting fresh, but nature never calls it quits. It's a process, not a thinking entity. In any stable population there will be variances that have neither a benefit or cost until environmental pressures force it to "select" the most appropriate. You have to look at a longer timescale to see the adaptations take hold. You could say species dying out is calling it quits in a way, but evolution encompasses everything not just the extinct - but I don't think that's what you meant.
thrown_22 4y ago
Bullets also fly 30,000 times faster than bumble bees. We're not about to use them for pollination though.
- visarga 4y ago
  Computers calculate billions of times faster than the brain, what's your point?
  - thrown_22 4y ago
    That computers aren't brains.
    - IIAOPSW 4y ago
      A brain can certainly memorize and follow a fixed set of rules to update the tape on a Turing machine. Therefore the brain is at least a Universal Computer. Furthermore there is nothing in the domain of sensory-motor tasks that is in principle beyond computer controlled robotics. So then praytell, what can a brain do that a computer can't? Careful. The current answers to that question live on a small island that's been eroding at an alarming pace. Some project that the island will be totally washed away in our lifetime. You sure you want to build there?
      - thrown_22 4y ago
        >Careful. The current answers to that question live on a small island that's been eroding at an alarming pace. Some project that the island will be totally washed away in our lifetime. You sure you want to build there? Yeah.
        IIAOPSW 4y ago
        Elaborate why?
      - samus 4y ago
        The answer to that question is capabilities usually attributed to Strong AI. Depending on who you ask, we are currently either 10 years or forever away from that, which pretty much means the same.
POiNTx 4y ago
I know these don't map 1-to-1 on real natural synapses, but let's say you can magically replace all your natural synapses in a brain with this. What would be the experienced effect? Would you be able to think faster, or would perceived time slow down? Or something else?
- IIAOPSW 4y ago
  Here's a partial answer. The perception of time always happens at one second per second.
- miovoid 4y ago
  Imagine a game world, where all your opponents 30000 times slower, 30000 times less energy efficient. Time and energy central resources in this game, which cannot be faked anyhow. Even if nuclear fusion or antimatter energy sources are everywhere.
  - manholio 4y ago
    Sounds like a very boring game. The two races are basically distinct, just like the human race is distinct from geology. For all we know, Earth's geology might be sentient already.
- netsharc 4y ago
  I wonder if a human's "refresh rate" has been evolutionary optimized for its environment. Maybe there's no need to evaluate things 30000x faster if our bodies can't react that fast. I guess it would make for faster reaction times, e.g. when driving: driver in front of you hits the brakes, the vision signal enters your brain with the same delay (are optic nerves synapses too?), but I guess the difference is your brain would be able to process the meaning of the change (the red lights coming on, the car getting larger in your vision) a lot faster. The brain then sends a signal for the foot to react, but the amount of time it will take for the foot to move will be the same as normal since the signal takes the same time to reach the foot. I imagine at 30000x, you would notice that it's taking very very long for the foot to move. To notice it would mean to have feedback from the skin inside your socks/pants that the foot/leg is moving against your sock/pants, and the brain will probably think that feedback is taking forever after it sent the signal.
yarg 4y ago
What're the thermal dissipation requirements? How densly is it possible to structure them and still provide adequate cooling?
- incrudible 4y ago
  The cooling problem is not thermal dissipation, but rather getting to superconductivity, which generally requires temperatures near absolute zero.
dschuetz 4y ago
I'm not sure whatever for they need such nanowire synapses, because in real/wet neural networks the timings of neural firing, delay and latency is as essential as the synapse itself. How fast a wet synapse works is modulated, some are isolated from modulation. I don't think a new synthetic synapse being 30.000x faster will do anything good if the whole system is not up to par with a wet neural system.

The piece is chock full of interesting findings, using terms biologists routinely use. But, do those terms they use, e.g. neuron, synapse mean the same thing they do for biologists? For instances, we know that synapses can be one of excitatory or inhibitory, and we know that neurons are bathed in a wash of hormones. Neurons make hormones which serve other functions throughout the brain. For instance "Neuron-Derived Estrogen Regulates Synaptic Plasticity and Memory" [1]. How does the linked work stack up against that? [1] https://www.jneurosci.org/content/39/15/2792 https://www.jneurosci.org/content/39/15/2792

superkuh 4y ago
All that, more, and it seems like every computational biology analogy just completely forgets about the most common cell type in the brain: astrocytes. And then there are things like axo-axonal transmission that totally blow up the simple models, https://www.cell.com/neuron/fulltext/S0896-6273(22)00656-0 https://www.cell.com/neuron/fulltext/S0896-6273(22)00656-0
- ad404b8a372f2b9 4y ago
  Can you elaborate on how the axo-axonal transmissions blow up our simple models? I couldn't understand anything from that summary. Would it be the equivalent of edges communicating between each other in artifical neural networks?
- bismuthcrystal 4y ago
  Biologists just won't allows us to have any fun. It is always this kind of rhetoric: "what, are you modeling the brain without considering the influence of <insert obscure type of cell> on the hormone regulated blood flow around ion pump circuits during chinese new year neuron firing patterns? you are obviously bounded to fail..." This whole AI field keeps on failing because people like to overthink things. Did Michelangelo need to know molecular chemistry to make sculptures? Why do people pretend there is no artistic component to building AI? Rant finished.
  - bee_rider 4y ago
    I don't think this is the case; the whole field of AI seems pretty healthy at the moment, not failing, and not all that worried about the inaccuracy of their model (It's only a model /Patsy -- but it can still host the whole song-and-dance).
  - IIAOPSW 4y ago
    While I agree there is a large contingent of people who seem to believe every detail of real neurons are necessary to cognition (up to and including the atoms), and while I tend to agree those people are way off the mark about which details are actually relevant to the algorithms of cognition versus which are specific to natures implementation thereof, I also think the existing AI field goes too far in the other direction and oversimplifies more than it can get away with. So far we've been able to gloss over our mistake through the raw brute force of voluminous training data and GPU power. It works in the same way using a hammer to drive a screw into a wall works. Sure, we can do it to an extent, but there's a much better way. We need to figure out how to use a screwdriver. And by screwdriver I mean slightly more sophisticated artificial neuron.
orbifold 4y ago
Short answer is no. The field is full with tenuous analogies. Then again „Neural Networks“ are also at best metaphorically related. More accurate existing Neuron models are actually also plagued by lots of limitations among them that they are typically implemented in 3 ancient domain specific languages with lots and lots of hardcoded constants copied from research papers.
- a-dub 4y ago
  spiking neural networks are interesting though, and new computational substrates that allow for experiments at larger scales could produce some interesting results. today's sum n' squash (sometimes not even squash) graph networks were just kind of a curiosity before gpus turned them into a new very successful computational paradigm. maybe we'll see something similar with these high element count optical spiking graphs, even if they aren't great approximations of the real biology. i like to think that a new analog computational substrate (or mixed analog and digital system) will be what drives the next leap in machine computation.
  - mjhay 4y ago
    I'm excited to see where spiking NNs go. Something like that is needed to progress now that conventional NNs on GPUs are pretty much tapped out (in terms of non-incremental advances) from their power consumption and the end of Moore's law. Things really do need to be more hardware-efficient.
mdp2021 4y ago
> But, do those terms they use, e.g. neuron, synapse mean the same thing they do for biologists They are not meant to. This is not "brain simulation" or similar - which exists, but is a different matter. This context is instead about neuromorphic computing, as hardware implementation of components for Artificial Neural Networks. And results seem to be remarkable: > They calculated that the synapses are capable of spike rates exceeding 10 million hertz while consuming roughly 33 attojoules of power per synaptic event (an attojoule is 10-18 of a joule) The comparison with biological neuro-transmission is just indicative - for trivia, for curiosity. -- Edit: on the contrary, these devices aim to be in a way simpler than ANN's neurons (far from aiming to be as complex as cerebral neurons): > By only rarely firing spikes, these devices shuffle around much less data than typical artificial neural networks and, in principle, require much less power and communication bandwidth That is because the underlying aim is to achieve using a single photon for communication, with an immediate potential practical use in ANNs.
- bee_rider 4y ago
  We should also note that the article only references the over-dramatic comparison to biological neurons in passing in the first paragraph (wonder if it is an "author gets it, but doesn't get to pick the title" type problem).
dr_dshiv 4y ago
The main idea is oscillatory neurodynamics. When you have a bunch of tuned oscillators, you can produce incredible computational capacity.

raziel2701 4y ago

These work only at a few Kelvin because they need superconducting nanowires. Will this type of stuff ever leave the lab?

TheRealPomax 4y ago

Not exactly impressive, nature is famous for calling it quits once something works well enough. Anyone who wants to can create a non-biological neuron/axion connection that runs orders of magnitude faster. The real question is "but does it actually do something" because that's where nature shines pretty well.

miovoid 4y ago
Turing and Church said yes.
- TheRealPomax 4y ago
  Said yes to... what? That artificial neurons can do something useful? Because those weren't a thing when they were scienceing it up.
yourapostasy 4y ago
> ...nature is famous for calling it quits once something works well enough... Great, so we're just bags of meat sashaying around with loads of technical debt baggage via the slapdash coding delivered by nature in a multi-billion year project. The ongoing result of always picking good and cheap over fast.
- TheRealPomax 4y ago
  you bet, nature looked at min-max strategies and went "k but what if min-med instead?" then did that for a few billion years.
TeacherTortoise 4y ago
> nature is famous for calling it quits Yes, evolution builds upon what went before rather than starting fresh, but nature never calls it quits. It's a process, not a thinking entity. In any stable population there will be variances that have neither a benefit or cost until environmental pressures force it to "select" the most appropriate. You have to look at a longer timescale to see the adaptations take hold. You could say species dying out is calling it quits in a way, but evolution encompasses everything not just the extinct - but I don't think that's what you meant.

thrown_22 4y ago

Bullets also fly 30,000 times faster than bumble bees. We're not about to use them for pollination though.

visarga 4y ago
Computers calculate billions of times faster than the brain, what's your point?
- thrown_22 4y ago
  That computers aren't brains.
  - IIAOPSW 4y ago
    A brain can certainly memorize and follow a fixed set of rules to update the tape on a Turing machine. Therefore the brain is at least a Universal Computer. Furthermore there is nothing in the domain of sensory-motor tasks that is in principle beyond computer controlled robotics. So then praytell, what can a brain do that a computer can't? Careful. The current answers to that question live on a small island that's been eroding at an alarming pace. Some project that the island will be totally washed away in our lifetime. You sure you want to build there?
    - thrown_22 4y ago
      >Careful. The current answers to that question live on a small island that's been eroding at an alarming pace. Some project that the island will be totally washed away in our lifetime. You sure you want to build there? Yeah.
      - IIAOPSW 4y ago
        Elaborate why?
    - samus 4y ago
      The answer to that question is capabilities usually attributed to Strong AI. Depending on who you ask, we are currently either 10 years or forever away from that, which pretty much means the same.

POiNTx 4y ago

I know these don't map 1-to-1 on real natural synapses, but let's say you can magically replace all your natural synapses in a brain with this. What would be the experienced effect? Would you be able to think faster, or would perceived time slow down? Or something else?

IIAOPSW 4y ago
Here's a partial answer. The perception of time always happens at one second per second.
miovoid 4y ago
Imagine a game world, where all your opponents 30000 times slower, 30000 times less energy efficient. Time and energy central resources in this game, which cannot be faked anyhow. Even if nuclear fusion or antimatter energy sources are everywhere.
- manholio 4y ago
  Sounds like a very boring game. The two races are basically distinct, just like the human race is distinct from geology. For all we know, Earth's geology might be sentient already.
netsharc 4y ago
I wonder if a human's "refresh rate" has been evolutionary optimized for its environment. Maybe there's no need to evaluate things 30000x faster if our bodies can't react that fast. I guess it would make for faster reaction times, e.g. when driving: driver in front of you hits the brakes, the vision signal enters your brain with the same delay (are optic nerves synapses too?), but I guess the difference is your brain would be able to process the meaning of the change (the red lights coming on, the car getting larger in your vision) a lot faster. The brain then sends a signal for the foot to react, but the amount of time it will take for the foot to move will be the same as normal since the signal takes the same time to reach the foot. I imagine at 30000x, you would notice that it's taking very very long for the foot to move. To notice it would mean to have feedback from the skin inside your socks/pants that the foot/leg is moving against your sock/pants, and the brain will probably think that feedback is taking forever after it sent the signal.

yarg 4y ago

What're the thermal dissipation requirements? How densly is it possible to structure them and still provide adequate cooling?

incrudible 4y ago
The cooling problem is not thermal dissipation, but rather getting to superconductivity, which generally requires temperatures near absolute zero.

dschuetz 4y ago

I'm not sure whatever for they need such nanowire synapses, because in real/wet neural networks the timings of neural firing, delay and latency is as essential as the synapse itself. How fast a wet synapse works is modulated, some are isolated from modulation. I don't think a new synthetic synapse being 30.000x faster will do anything good if the whole system is not up to par with a wet neural system.