14 ms·

How Computationally Complex Is a Single Neuron?

188 points by theafh 5y ago

0-_-0 5y ago
A more interesting angle would be "how complex is the genetic description of a neuron"? Or "How many degrees of freedom are there in the genetic description of a neuron"?
- luc4sdreyer 5y ago
  Why? What would that tell us about the computational complexity of a neuron?
  - hall0ween 5y ago
    To me, it seems like going up a layer in biological "abstraction"...
    - JulianMorrison 5y ago
      Because the process that constructs a body part is only partially genetic, you'd be leaving out meaningful data. Mistaken idea: genes are like a program in code that defines what biology does. More accurate idea: genes are like the NVRAM of a running program that has run continuously with in-place updates for 4 billion years.
      - ianmcgowan 5y ago
        What an amazing comment, that just blew my mind. Talk about a paradigm shift.
      - spiralx 5y ago
        > More accurate idea: genes are like the NVRAM of a running program that has run continuously with in-place updates for 4 billion years. They also represent the NVRAM of the compiler that builds both the program and the compiler itself :)
    - burnished 5y ago
      So, we know that human-like general intelligence is possible because we display it right? We're pretty sure it has something to do with that soft stuff inside our brain-baskets, and how that soft stuff is interconnected. We've found computational primitives that we call neurons, so the premise goes, if we can understand and copy how it works we can engineer intelligence. Right? So going to genetics, while interesting and worthy, isn't really going up a level of 'abstraction' because we're trying to get a systems level overview here. It is a process somewhat like studying a CPU by shocking prongs and occasionally slicing it real thin to see inside it.
  - kittiepryde 5y ago
    It might give us an idea of how many types of models we would need to correspond to all genetically reasonable types of neurons. (my interpretation of the comment) For example, if there are like 8 free variables on neuron formation from the genetic blueprints, and then we find they produce reasonably different neuron types about 16 times over the range of that variable. We could end up with 4 billion possible models that we would need to simulate all possible neurons; then I guess you'd want to reduce those by saying that these two were like these other two and starting condensing them down; I'm not sure where this would go, but maybe we would find something cool or find a new insight by figuring out all possibilities
    - causi 5y ago
      The difference between being able to describe a neuron's behavior and being able to simulate that behavior is the difference between being able to say "ten trillion trillion PN junctions" and being able to build a working computer that fits that description.
  - 0-_-0 5y ago
    This paper has found that you need 1000 artificial neurons to simulate a real one. This doesn't say much about the actual complexity of a neuron though, e.g. I'm sure you need many neurons to simulate a Fourier transform, even though the Fourier transform only has a few mathematical terms (in other words, the true complexity of a FT is much lower). What would tell us about its actual complexity though is if we knew how many "variables" the formula describing a single neuron has, which should be approximated by how many degrees of freedom in nature does its genetic description have.
    - smegger001 5y ago
      i wonder if its the case that neurons are something of a Turing Tarpit doing something like implementing a Turing Machine but only using a limited number of operators other basic operation having to be implemented from the ones present (ie making a on XOR out of NAND gates for example) just highly optimized for a particular use-case.
- burnished 5y ago
  That isn't really pertinent or I'd argue interesting in the context of this article, at all! It might be helpful to know that (some | many | most) people think of biological neurons as being sort of poor-quality artificial ones, or at least that there is a direct correspondence between an organic neuron and an artificial one. This paper is making the argument that a given biological neuron is more like a network of artificial neurons, which is interesting and important because a lot of research into intelligence right now is going into making 'biologically plausible' models and testing those. If their results are essentially that each neuron has the sophistication of a network then say a future model of a cortical 'cell' (a cluster of neurons we think have a purpose as a group) should take this into account. Genetics are interesting but neurologically we're still at the phase of knowledge where we are making up wild-ass conjectures and shooting holes in them for want of anything better to be doing.
datameta 5y ago
Relevant read for the curious: "The thermodynamic efficiency of computations made in cells across the range of life" https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5686401/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5686401/
- Koshkin 5y ago
  Looks like what the Quanta article refers to is not so much "computations made in cells" as "how computationally complex a faithful model of a neuron would be."
  - AlexCoventry 5y ago
    Based on the Quanta article, they're only measuring complexity in terms of how complex a neural net needs to be to faithfully learn a neuron's output (presumably by SGD.) That's probably only a weak upper bound on the complexity, though. There may be more parsimonious ways to simulate the neuron which are expensive to represent in a neural net, or which are hard for SGD to learn.
yellow_lead 5y ago
It seems to be fairly complex. I hope we don't go down the path of building biological-artificial computers though.
- Koshkin 5y ago
  https://en.wikipedia.org/wiki/Biological_computing https://en.wikipedia.org/wiki/Biological_computing
- netizen-936824 5y ago
  Why not? As a someone in the field, I hope that we do.
  - datameta 5y ago
    Surely you recognize the plethora of ethical concerns behind this? Certainly there is much to be learned in terms of energy efficiency of computation but I'm undecided as to whether we should go down this path. If we do, I believe it should be regulated from the get-go or else it will be too late by the time the biological substrates reach animal-analogous intelligence. Perhaps if done for classical computing research without trialing machine learning we can avoid asking ourselves at what point the biological machine has sentience and deserves agency?
    - netizen-936824 5y ago
      I'm sorry, but I fail to see the ethical dilemma of creating an artificial biological computer.
    - sgillen 5y ago
      Do you believe that using a biological computer for research is ethically any different than an animal analogous intelligence in silicon?
      - datameta 5y ago
        I think we will have to navigate similar ethical straits in the development of both organic and inorganic sentience.
nicoffeine 5y ago
Anirban Bandyopadhyay, who has published a lot of research on the topic, has presented good evidence that microtubules have quantum properties. From one published paper: "We demonstrate that a single brain-neuron-extracted microtubule is a memory-switching element, whose hysteresis loss is nearly zero. Our study shows how a memory-state forms in the nanowire and how its protein arrangement symmetry is related to the conducting-state written in the device, thus, enabling it to store and process ∼500 distinct bits, with 2 pA resolution between 1 nA and 1 pA. Its random access memory is an analogue of flash memory switch used in a computer chip. Using scanning tunneling microscope imaging, we demonstrate how single proteins behave inside the nanowire when this 3.5 billion years old nanowire processes memory-bits."[1] As he discusses in an interview[2], the idea that the membrane is the only important part of the neuron is an idea from 1907 and is fundamentally incorrect. While he does not claim to prove Orch-OR and has his own theories, he is sure the internal structure of neurons with protofilaments of various sizes capable of up to terahertz frequencies has vital functionality that needs its place in neuroscience. There are about 5,000 microtubules per neuron. If these are quantum devices, we are many thousands or millions of years away from AGI, if it's even possible to achieve it without replicating those structures. [1] https://aip.scitation.org/doi/abs/10.1063/1.4793995 https://aip.scitation.org/doi/abs/10.1063/1.4793995 [2] https://www.closertotruth.com/series/quantum-physics-consciousness#video-48851 https://www.closertotruth.com/series/quantum-physics-conscio...
- Jweb_Guru 5y ago
  As someone who's extremely skeptical about the premise of AGI, I don't see how these results (if they are replicable) would imply anything much about it. Computers are not exactly storage-limited as it is, and latency and (cross-sectional) bandwidth to memory tend to be a lot more impactful than raw capacity in terms of impact on effective computational power. Besides that, neurons have a lot more stuff in them than just microtubules (since they're living cells), which gives us a lot of wiggle room to improve on them even if our actual circuits are potentially less space-efficient than the biological alternatives. IMO, the appeal of studying biological organisms for ideas for computation is a lot more about things like energy efficiency and robustness than pure performance.
sockpuppet_12 5y ago
It's amazing to live in a time where we have the technology to investigate all the incredibly well architected systems in molecular biology. The researchers must be having so much fun, I feel happy for them.
IIAOPSW 5y ago
I suspect not very complex at all. My rank speculation is that neurons are mostly a medium for brain waves and the real processing happens with the waves themselves. Looking for how the brain works in the neuron is like looking for how oceans work in the water molecule.
- brokencode 5y ago
  Where do you think these “brain waves” come from? Brain activity is the result of the interaction of neurons and external stimuli. Just as transistors are fundamental to processors, neurons are fundamental to brains, and it is important that we understand these basic building blocks if we ever hope to understand the brain and intelligence as a whole.
  - IIAOPSW 5y ago
    >Where do you think these “brain waves” come from? Propagation delay and circular loops. No it doesn't have to be "external" stimuli. There's no explicit input and output layer in how a real brain is constructed. That is an artificial constraint put on artificial neural nets to make a programmers job easier. A real brain can and likely does have self stimulating loops. >Just as transistors are fundamental to processors, neurons are fundamental to brains Transistors are also quite simple as individual units and the computational complexity comes about in how they are connected.
    - lebuffon 5y ago
      "A real brain can and likely does have self stimulating loops" This is what I understand causes epilepsy. At least when they are out of control.
- folli 5y ago
  Wow, that sounds very new-agey. You should also weave in a mention of quantum physics.
  - ordu 5y ago
causi 5y ago
A single neuron has somewhere on the order of 5x10^16 atoms in it. There's no rule that says it couldn't potentially use a large fraction of those atoms for its functionality. That's your upper bound on computational complexity.
- bckr 5y ago
  Yes, and we don't know how much of that possible computational complexity is required for producing our consciousness. However, if we are modeling particular functions e.g. motor control of the limbs, we can black box a lot of that complexity.
- qweqwweqwe-90i 5y ago
  That's not a informative upper bound. We already know most of those atoms are not contributing to it's computational complexity.
  - vixen99 5y ago
    its
  - jjtheblunt 5y ago
    why is that known?
  - acchow 5y ago
    How do we know this?
    - not2b 5y ago
      Read the article. They accurately modeled the biological neuron's behavior with about 1000 'neurons'. They didn't need quintillions of neurons. So clearly the computational complexity is vastly lower than the number of atoms or molecules.
      - jhayward 5y ago
question000 5y ago
Probably less complex then a 1, but more complex then a 0.
cblconfederate 5y ago
- The comparison here is between "A system of thousands of linear differential equations running in parallel" (The Hodkin-Huxley based modeling that is used to simulate the voltages of the biologically detailed model) and "A typical multi-layer neural network". One could probably calculate the computational complexity of the first and the second and assess how much it compressed the original function - The goal of artificial neural networks is not generally to simulate with 100% accuracy the firing of real neurons, but to perform their function. The results from ANN applications so far show impressive abilities that match equivalent (but specific) functions of hundreds of thousands of real brain neurons
coffeecat 5y ago
I'd be interested to learn how many biological neurons it takes to approximate one artificial neuron. Mathematically, over some finite interval, approximating y=sin(x) using a Taylor series, or approximating y=x**n using a Fourier series, can both require a large number of terms. But this doesn't imply that a sine/cosine function is more computationally complex than a monomial, or vice versa.
- PeterisP 5y ago
  The article asserts that on average they need ~1000 "neurons" to approximate one biological neuron. In the opposite direction, assuming we had manipulation techniques create new neurons, position them and stretch out axons arbitrarily and control synapse linking strength (which we can't now), then IMHO there could be a 1-to-1 correspondence; the same octopus neurons that McCulloch-Pitts model tried to describe could also each approximate one artificial neuron for inference. Of course, learning is a different issue, brains don't exactly do back-propagation and gradient descent.
- diskzero 5y ago
  Is there any correlation between a perceptron and a neuron? When I first encountered the concept of a perceptron, I was told Minsky and Papert modeled the perceptron on the biology of a neuron. Is this correct? If so, how has this initial model stood up and we learn more about the physical structure of the neuron.
- selfish_gene 5y ago
  The answer is 1. You only need to put all your synapses at the soma and voila
jasonmar 5y ago
A single neuron has hundreds to hundreds of thousands of synapses and at each synapse there is a number of receptors and channels on the receptor end and neurotransmitter molecules and vesicles on the transmitter end. Synapses are far from the nucleus so there are reservoirs of mRNA hanging out waiting to be transcribed that needed to be staged there. There's epigenetic state affecting how much and what type of mRNA is produced. All of these are continuously affected by the combination inputs. How much of this needs to be included in the model? I'd like to see them comment on which parts of the original system they consider out of scope.
- netizen-936824 5y ago
  That's not even mentioning the fact that most receptors are quaternary structures in which the subunits can be switched out. This allows a single receptor to have many variations which means many variations on receptor behavior. Pardon my language, but the complexity is fucking mind boggling. It's beautiful.
  - amelius 5y ago
    But how much of that complexity is "observable"?
    - prox 5y ago
      You mean what is absolutely necessary?
      - amelius 5y ago
        Imagine you have a 2-input NAND gate, but for some reason it is implemented with 1000 transistors (perhaps for redundancy in case one of the transistors gets hit by a cosmic ray, or perhaps for other reasons). That gate still behaves the same, so for all an (external) observer could measure, it is a NAND-gate, which is a simple device. Internal complexity does not always mean external (observable) complexity.
joe_the_user 5y ago
How computationally complex is a transistor? I mean, all the ways that even a micro transistor interacts with voltages and heat make it potentially more complex than an AND/OR gate. But that's often what a given transistor is used for. I'm not saying all the complexity of a neuron doesn't matter or that neuron isn't much more complex than a physical chip piece - just that treating parts in isolation, if you don't understand the system, can very easily result in wrong conclusions.
- parsecs 5y ago
  Another part of the consideration is that transistors are abstracted away into functions like logic gates partially because it makes it easier for the designers. Neurons on the other hand can likely take advantage of all sorts of complex interactions and behaviors as the "maximum local complexity" is not an issue.
  - biofox 5y ago
    They certainly do take advantage of complex interactions. The perfect example of that is ephaptic coupling, where neurons interact with each other through the local electrical fields they induce -- and stuff like this is lost in modelling a single neuron: https://en.wikipedia.org/wiki/Ephaptic_coupling https://en.wikipedia.org/wiki/Ephaptic_coupling
- moron4hire 5y ago
  I remember reading about an experiment that used genetic algorithms to evolve FPGA layouts to create circuits that achieved some task while minimizing transistor count. I think it might have been NASA trying to develop smaller, more sensitive radios, but I'm not sure. IIRC, they found that they could evolve novel circuits that could perform the task with far fewer transistors than traditional methods. But, they couldn't figure out how to transcribe it to an IC outside of the FPGA, as the circuits would stop working. There were also disconnected sections of the FPGA that the GA had evolved that didn't appear to contribute to the output, but couldn't be removed without again breaking the functionality. Obviously, there were some kind of crosstalk issues going on that the GA could minimax to a solution, but it was extremely brittle to change. EDIT: I was completely wrong about the source and application. Can't believe I was able to find it, because I was way off. This was it: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.9691 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.... Other reading indicates that this technique has been successfully applied in electronics design in the time since 1996. https://en.m.wikipedia.org/wiki/Evolvable_hardware https://en.m.wikipedia.org/wiki/Evolvable_hardware The NASA thing I was thinking of was this: https://en.m.wikipedia.org/wiki/Evolved_antenna https://en.m.wikipedia.org/wiki/Evolved_antenna
YeGoblynQueenne 5y ago
Sorry but this is the dumbest thing I've heard in the entire internet today. So basically what they did was they trained a deep neural net to predict the behaviour of a simulated rat neuron. Then they claimed that the number of layers and units in the deep neural net is somehow a measure of the complexity of a "biological neuron". First of all, they didn't try to model the behaviour of a biological neuron, only of a model of a biological neuron, and an incomplete model at that. The article points that out itself. More importantly, the claim rests on the assumption that a deep neural net architecture is somehow a sensible measure of the complexity of a completely unrelated process (as far as I can tell, deep neural nets are not in anyway related to simulated rat neurons). But this is a big, huge, gigantic assumption. Suppose I train a bunch of humans to predict the behaviour of a simulated rat neuron. Suppose I find out that I need to train at least 1000 humans to accurately predict the simulated rat neuron's behaviour. What does that mean? That 1000 human brains are as complex as a simulated rat neuron? Come to that- can a single human brain predict the behaviour of a simulated rat neuron? I'm prepared to wager that, no, it can't, because human brains are rubbish at prediction tasks like that. What does that mean? That a simulated rat neuron is more complex than a human with an entire human brain? The entire premise of the work is absurd nonsense and it probably only got a mention in Quanta because it's something released by DeepMind. Irritating.
- orbifold 5y ago
  Science is in large part also about story telling, especially in some areas. This is a good story, even if the details make it much less spectacular. They also produced very nice visualisations, which blew up on Twitter. All of these things are something to learn from as a scientist.
  - dekhn 5y ago
    The lesson is that while people say they make data-driven decisions, scientists like everybody else just goes for the narrative with the prettiest picture that also looks like what people think is already true.
marco_craveiro 5y ago
Thanks for sharing, very interesting. Haven't read the paper yet, but it seems promising as it mentions Hawkins and Subotai, from Numenta, who have been arguing something along these lines for many years. It is available in biorxiv [1]. [1] https://www.biorxiv.org/content/10.1101/613141v2.full.pdf https://www.biorxiv.org/content/10.1101/613141v2.full.pdf
- marco_craveiro 5y ago
  OK I am still parsing the pre-print, but their approach seems very interesting. And equally interesting would be to see what happens if you apply HTM [1] to the same inputs model, e.g.: > We next applied our paradigm to a morphologically and electrically complex detailed biophysical compartmental model of a 3D reconstructed layer 5 cortical pyramidal cell (L5PC) from rat somatosensory cortex (Fig. 2A). The model is equipped with complex nonlinear membrane properties, a somatic spike generation mechanism and an excitable apical nexus capable of generating calcium spikes [1] https://github.com/htm-community/htm.core https://github.com/htm-community/htm.core

188 points by theafh 5y ago

A more interesting angle would be "how complex is the genetic description of a neuron"? Or "How many degrees of freedom are there in the genetic description of a neuron"?

luc4sdreyer 5y ago
Why? What would that tell us about the computational complexity of a neuron?
- hall0ween 5y ago
  To me, it seems like going up a layer in biological "abstraction"...
  - JulianMorrison 5y ago
    Because the process that constructs a body part is only partially genetic, you'd be leaving out meaningful data. Mistaken idea: genes are like a program in code that defines what biology does. More accurate idea: genes are like the NVRAM of a running program that has run continuously with in-place updates for 4 billion years.
    - ianmcgowan 5y ago
      What an amazing comment, that just blew my mind. Talk about a paradigm shift.
    - spiralx 5y ago
      > More accurate idea: genes are like the NVRAM of a running program that has run continuously with in-place updates for 4 billion years. They also represent the NVRAM of the compiler that builds both the program and the compiler itself :)
  - burnished 5y ago
    So, we know that human-like general intelligence is possible because we display it right? We're pretty sure it has something to do with that soft stuff inside our brain-baskets, and how that soft stuff is interconnected. We've found computational primitives that we call neurons, so the premise goes, if we can understand and copy how it works we can engineer intelligence. Right? So going to genetics, while interesting and worthy, isn't really going up a level of 'abstraction' because we're trying to get a systems level overview here. It is a process somewhat like studying a CPU by shocking prongs and occasionally slicing it real thin to see inside it.
- kittiepryde 5y ago
  It might give us an idea of how many types of models we would need to correspond to all genetically reasonable types of neurons. (my interpretation of the comment) For example, if there are like 8 free variables on neuron formation from the genetic blueprints, and then we find they produce reasonably different neuron types about 16 times over the range of that variable. We could end up with 4 billion possible models that we would need to simulate all possible neurons; then I guess you'd want to reduce those by saying that these two were like these other two and starting condensing them down; I'm not sure where this would go, but maybe we would find something cool or find a new insight by figuring out all possibilities
  - causi 5y ago
    The difference between being able to describe a neuron's behavior and being able to simulate that behavior is the difference between being able to say "ten trillion trillion PN junctions" and being able to build a working computer that fits that description.
- 0-_-0 5y ago
  This paper has found that you need 1000 artificial neurons to simulate a real one. This doesn't say much about the actual complexity of a neuron though, e.g. I'm sure you need many neurons to simulate a Fourier transform, even though the Fourier transform only has a few mathematical terms (in other words, the true complexity of a FT is much lower). What would tell us about its actual complexity though is if we knew how many "variables" the formula describing a single neuron has, which should be approximated by how many degrees of freedom in nature does its genetic description have.
  - smegger001 5y ago
    i wonder if its the case that neurons are something of a Turing Tarpit doing something like implementing a Turing Machine but only using a limited number of operators other basic operation having to be implemented from the ones present (ie making a on XOR out of NAND gates for example) just highly optimized for a particular use-case.
burnished 5y ago
That isn't really pertinent or I'd argue interesting in the context of this article, at all! It might be helpful to know that (some | many | most) people think of biological neurons as being sort of poor-quality artificial ones, or at least that there is a direct correspondence between an organic neuron and an artificial one. This paper is making the argument that a given biological neuron is more like a network of artificial neurons, which is interesting and important because a lot of research into intelligence right now is going into making 'biologically plausible' models and testing those. If their results are essentially that each neuron has the sophistication of a network then say a future model of a cortical 'cell' (a cluster of neurons we think have a purpose as a group) should take this into account. Genetics are interesting but neurologically we're still at the phase of knowledge where we are making up wild-ass conjectures and shooting holes in them for want of anything better to be doing.

datameta 5y ago

Relevant read for the curious: "The thermodynamic efficiency of computations made in cells across the range of life" https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5686401/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5686401/

Koshkin 5y ago
Looks like what the Quanta article refers to is not so much "computations made in cells" as "how computationally complex a faithful model of a neuron would be."
- AlexCoventry 5y ago
  Based on the Quanta article, they're only measuring complexity in terms of how complex a neural net needs to be to faithfully learn a neuron's output (presumably by SGD.) That's probably only a weak upper bound on the complexity, though. There may be more parsimonious ways to simulate the neuron which are expensive to represent in a neural net, or which are hard for SGD to learn.

yellow_lead 5y ago

It seems to be fairly complex. I hope we don't go down the path of building biological-artificial computers though.

Koshkin 5y ago
https://en.wikipedia.org/wiki/Biological_computing https://en.wikipedia.org/wiki/Biological_computing
netizen-936824 5y ago
Why not? As a someone in the field, I hope that we do.
- datameta 5y ago
  Surely you recognize the plethora of ethical concerns behind this? Certainly there is much to be learned in terms of energy efficiency of computation but I'm undecided as to whether we should go down this path. If we do, I believe it should be regulated from the get-go or else it will be too late by the time the biological substrates reach animal-analogous intelligence. Perhaps if done for classical computing research without trialing machine learning we can avoid asking ourselves at what point the biological machine has sentience and deserves agency?
  - netizen-936824 5y ago
    I'm sorry, but I fail to see the ethical dilemma of creating an artificial biological computer.
  - sgillen 5y ago
    Do you believe that using a biological computer for research is ethically any different than an animal analogous intelligence in silicon?
    - datameta 5y ago
      I think we will have to navigate similar ethical straits in the development of both organic and inorganic sentience.

nicoffeine 5y ago

Anirban Bandyopadhyay, who has published a lot of research on the topic, has presented good evidence that microtubules have quantum properties. From one published paper: "We demonstrate that a single brain-neuron-extracted microtubule is a memory-switching element, whose hysteresis loss is nearly zero. Our study shows how a memory-state forms in the nanowire and how its protein arrangement symmetry is related to the conducting-state written in the device, thus, enabling it to store and process ∼500 distinct bits, with 2 pA resolution between 1 nA and 1 pA. Its random access memory is an analogue of flash memory switch used in a computer chip. Using scanning tunneling microscope imaging, we demonstrate how single proteins behave inside the nanowire when this 3.5 billion years old nanowire processes memory-bits."[1] As he discusses in an interview[2], the idea that the membrane is the only important part of the neuron is an idea from 1907 and is fundamentally incorrect. While he does not claim to prove Orch-OR and has his own theories, he is sure the internal structure of neurons with protofilaments of various sizes capable of up to terahertz frequencies has vital functionality that needs its place in neuroscience. There are about 5,000 microtubules per neuron. If these are quantum devices, we are many thousands or millions of years away from AGI, if it's even possible to achieve it without replicating those structures. [1] https://aip.scitation.org/doi/abs/10.1063/1.4793995 https://aip.scitation.org/doi/abs/10.1063/1.4793995 [2] https://www.closertotruth.com/series/quantum-physics-consciousness#video-48851 https://www.closertotruth.com/series/quantum-physics-conscio...

Jweb_Guru 5y ago
As someone who's extremely skeptical about the premise of AGI, I don't see how these results (if they are replicable) would imply anything much about it. Computers are not exactly storage-limited as it is, and latency and (cross-sectional) bandwidth to memory tend to be a lot more impactful than raw capacity in terms of impact on effective computational power. Besides that, neurons have a lot more stuff in them than just microtubules (since they're living cells), which gives us a lot of wiggle room to improve on them even if our actual circuits are potentially less space-efficient than the biological alternatives. IMO, the appeal of studying biological organisms for ideas for computation is a lot more about things like energy efficiency and robustness than pure performance.

sockpuppet_12 5y ago

It's amazing to live in a time where we have the technology to investigate all the incredibly well architected systems in molecular biology. The researchers must be having so much fun, I feel happy for them.

IIAOPSW 5y ago

I suspect not very complex at all. My rank speculation is that neurons are mostly a medium for brain waves and the real processing happens with the waves themselves. Looking for how the brain works in the neuron is like looking for how oceans work in the water molecule.

brokencode 5y ago
Where do you think these “brain waves” come from? Brain activity is the result of the interaction of neurons and external stimuli. Just as transistors are fundamental to processors, neurons are fundamental to brains, and it is important that we understand these basic building blocks if we ever hope to understand the brain and intelligence as a whole.
- IIAOPSW 5y ago
  >Where do you think these “brain waves” come from? Propagation delay and circular loops. No it doesn't have to be "external" stimuli. There's no explicit input and output layer in how a real brain is constructed. That is an artificial constraint put on artificial neural nets to make a programmers job easier. A real brain can and likely does have self stimulating loops. >Just as transistors are fundamental to processors, neurons are fundamental to brains Transistors are also quite simple as individual units and the computational complexity comes about in how they are connected.
  - lebuffon 5y ago
    "A real brain can and likely does have self stimulating loops" This is what I understand causes epilepsy. At least when they are out of control.
folli 5y ago
Wow, that sounds very new-agey. You should also weave in a mention of quantum physics.
- ordu 5y ago

causi 5y ago

A single neuron has somewhere on the order of 5x10^16 atoms in it. There's no rule that says it couldn't potentially use a large fraction of those atoms for its functionality. That's your upper bound on computational complexity.

bckr 5y ago
Yes, and we don't know how much of that possible computational complexity is required for producing our consciousness. However, if we are modeling particular functions e.g. motor control of the limbs, we can black box a lot of that complexity.
qweqwweqwe-90i 5y ago
That's not a informative upper bound. We already know most of those atoms are not contributing to it's computational complexity.
- vixen99 5y ago
  its
- jjtheblunt 5y ago
  why is that known?
- acchow 5y ago
  How do we know this?
  - not2b 5y ago
    Read the article. They accurately modeled the biological neuron's behavior with about 1000 'neurons'. They didn't need quintillions of neurons. So clearly the computational complexity is vastly lower than the number of atoms or molecules.
    - jhayward 5y ago

question000 5y ago

Probably less complex then a 1, but more complex then a 0.

cblconfederate 5y ago

- The comparison here is between "A system of thousands of linear differential equations running in parallel" (The Hodkin-Huxley based modeling that is used to simulate the voltages of the biologically detailed model) and "A typical multi-layer neural network". One could probably calculate the computational complexity of the first and the second and assess how much it compressed the original function - The goal of artificial neural networks is not generally to simulate with 100% accuracy the firing of real neurons, but to perform their function. The results from ANN applications so far show impressive abilities that match equivalent (but specific) functions of hundreds of thousands of real brain neurons

coffeecat 5y ago

I'd be interested to learn how many biological neurons it takes to approximate one artificial neuron. Mathematically, over some finite interval, approximating y=sin(x) using a Taylor series, or approximating y=x**n using a Fourier series, can both require a large number of terms. But this doesn't imply that a sine/cosine function is more computationally complex than a monomial, or vice versa.

PeterisP 5y ago
The article asserts that on average they need ~1000 "neurons" to approximate one biological neuron. In the opposite direction, assuming we had manipulation techniques create new neurons, position them and stretch out axons arbitrarily and control synapse linking strength (which we can't now), then IMHO there could be a 1-to-1 correspondence; the same octopus neurons that McCulloch-Pitts model tried to describe could also each approximate one artificial neuron for inference. Of course, learning is a different issue, brains don't exactly do back-propagation and gradient descent.
diskzero 5y ago
Is there any correlation between a perceptron and a neuron? When I first encountered the concept of a perceptron, I was told Minsky and Papert modeled the perceptron on the biology of a neuron. Is this correct? If so, how has this initial model stood up and we learn more about the physical structure of the neuron.
selfish_gene 5y ago
The answer is 1. You only need to put all your synapses at the soma and voila

jasonmar 5y ago

A single neuron has hundreds to hundreds of thousands of synapses and at each synapse there is a number of receptors and channels on the receptor end and neurotransmitter molecules and vesicles on the transmitter end. Synapses are far from the nucleus so there are reservoirs of mRNA hanging out waiting to be transcribed that needed to be staged there. There's epigenetic state affecting how much and what type of mRNA is produced. All of these are continuously affected by the combination inputs. How much of this needs to be included in the model? I'd like to see them comment on which parts of the original system they consider out of scope.

netizen-936824 5y ago
That's not even mentioning the fact that most receptors are quaternary structures in which the subunits can be switched out. This allows a single receptor to have many variations which means many variations on receptor behavior. Pardon my language, but the complexity is fucking mind boggling. It's beautiful.
- amelius 5y ago
  But how much of that complexity is "observable"?
  - prox 5y ago
    You mean what is absolutely necessary?
    - amelius 5y ago
      Imagine you have a 2-input NAND gate, but for some reason it is implemented with 1000 transistors (perhaps for redundancy in case one of the transistors gets hit by a cosmic ray, or perhaps for other reasons). That gate still behaves the same, so for all an (external) observer could measure, it is a NAND-gate, which is a simple device. Internal complexity does not always mean external (observable) complexity.

joe_the_user 5y ago

How computationally complex is a transistor? I mean, all the ways that even a micro transistor interacts with voltages and heat make it potentially more complex than an AND/OR gate. But that's often what a given transistor is used for. I'm not saying all the complexity of a neuron doesn't matter or that neuron isn't much more complex than a physical chip piece - just that treating parts in isolation, if you don't understand the system, can very easily result in wrong conclusions.

parsecs 5y ago
Another part of the consideration is that transistors are abstracted away into functions like logic gates partially because it makes it easier for the designers. Neurons on the other hand can likely take advantage of all sorts of complex interactions and behaviors as the "maximum local complexity" is not an issue.
- biofox 5y ago
  They certainly do take advantage of complex interactions. The perfect example of that is ephaptic coupling, where neurons interact with each other through the local electrical fields they induce -- and stuff like this is lost in modelling a single neuron: https://en.wikipedia.org/wiki/Ephaptic_coupling https://en.wikipedia.org/wiki/Ephaptic_coupling
moron4hire 5y ago
I remember reading about an experiment that used genetic algorithms to evolve FPGA layouts to create circuits that achieved some task while minimizing transistor count. I think it might have been NASA trying to develop smaller, more sensitive radios, but I'm not sure. IIRC, they found that they could evolve novel circuits that could perform the task with far fewer transistors than traditional methods. But, they couldn't figure out how to transcribe it to an IC outside of the FPGA, as the circuits would stop working. There were also disconnected sections of the FPGA that the GA had evolved that didn't appear to contribute to the output, but couldn't be removed without again breaking the functionality. Obviously, there were some kind of crosstalk issues going on that the GA could minimax to a solution, but it was extremely brittle to change. EDIT: I was completely wrong about the source and application. Can't believe I was able to find it, because I was way off. This was it: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.9691 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.... Other reading indicates that this technique has been successfully applied in electronics design in the time since 1996. https://en.m.wikipedia.org/wiki/Evolvable_hardware https://en.m.wikipedia.org/wiki/Evolvable_hardware The NASA thing I was thinking of was this: https://en.m.wikipedia.org/wiki/Evolved_antenna https://en.m.wikipedia.org/wiki/Evolved_antenna

YeGoblynQueenne 5y ago

Sorry but this is the dumbest thing I've heard in the entire internet today. So basically what they did was they trained a deep neural net to predict the behaviour of a simulated rat neuron. Then they claimed that the number of layers and units in the deep neural net is somehow a measure of the complexity of a "biological neuron". First of all, they didn't try to model the behaviour of a biological neuron, only of a model of a biological neuron, and an incomplete model at that. The article points that out itself. More importantly, the claim rests on the assumption that a deep neural net architecture is somehow a sensible measure of the complexity of a completely unrelated process (as far as I can tell, deep neural nets are not in anyway related to simulated rat neurons). But this is a big, huge, gigantic assumption. Suppose I train a bunch of humans to predict the behaviour of a simulated rat neuron. Suppose I find out that I need to train at least 1000 humans to accurately predict the simulated rat neuron's behaviour. What does that mean? That 1000 human brains are as complex as a simulated rat neuron? Come to that- can a single human brain predict the behaviour of a simulated rat neuron? I'm prepared to wager that, no, it can't, because human brains are rubbish at prediction tasks like that. What does that mean? That a simulated rat neuron is more complex than a human with an entire human brain? The entire premise of the work is absurd nonsense and it probably only got a mention in Quanta because it's something released by DeepMind. Irritating.

orbifold 5y ago
Science is in large part also about story telling, especially in some areas. This is a good story, even if the details make it much less spectacular. They also produced very nice visualisations, which blew up on Twitter. All of these things are something to learn from as a scientist.
- dekhn 5y ago
  The lesson is that while people say they make data-driven decisions, scientists like everybody else just goes for the narrative with the prettiest picture that also looks like what people think is already true.

marco_craveiro 5y ago

Thanks for sharing, very interesting. Haven't read the paper yet, but it seems promising as it mentions Hawkins and Subotai, from Numenta, who have been arguing something along these lines for many years. It is available in biorxiv [1]. [1] https://www.biorxiv.org/content/10.1101/613141v2.full.pdf https://www.biorxiv.org/content/10.1101/613141v2.full.pdf

marco_craveiro 5y ago
OK I am still parsing the pre-print, but their approach seems very interesting. And equally interesting would be to see what happens if you apply HTM [1] to the same inputs model, e.g.: > We next applied our paradigm to a morphologically and electrically complex detailed biophysical compartmental model of a 3D reconstructed layer 5 cortical pyramidal cell (L5PC) from rat somatosensory cortex (Fig. 2A). The model is equipped with complex nonlinear membrane properties, a somatic spike generation mechanism and an excitable apical nexus capable of generating calcium spikes [1] https://github.com/htm-community/htm.core https://github.com/htm-community/htm.core