9 ms·

Communication costs are 35-fold computational costs in the human brain (2021)

151 points by ofou 4y ago

anotherhue 4y ago
intra-neuronal communication, not 'talking to people' (tiring as that may be).
- calculated 4y ago
  Thank you for this specification as the OP hasn't put it in the title. Before reading your comment I thought the article is about human interaction.
  - devonallie 4y ago
    Maybe it’s because I’ve been spending too much time thinking about packets, but I thought it was “digital” communication.
- xwdv 4y ago
  It’s crazy how I was willing to believe that this is what the article was talking about without even opening it. Authors should be more specific, a lot of misinformation could be spread through misleading titles.
  - nanofortnight 4y ago
    "Communication in the human cortex" seems pretty clear. The abstract is there for an extended summary if one doesn't wish to read the whole article. One cannot possibly understand the nuances of something without reading it; I do not feel knowledge gained through reading titles of articles is usable knowledge. Perhaps one should consider changing the habits of the reader rather than the author?
    - xwdv 4y ago
      A lot of that nuance would be quickly forgotten, but the info from a good title could last a long time.
- layer8 4y ago
  I believe you mean inter-neural communication? But, in consequence, by “computation” they must mean intra-neural computation? Because otherwise (i.e. for inter-neural computation) the “communication” would be an integral part of the computation. Or do they really mean that most of the energy cost of neural computation must be attributed to inter-neural communication?
mjburgess 4y ago
I have friends who are non-science PhDs and absolutely believe anything published in a science journal must be absolutely sound. Before I started reading I thought, "alas, its going to be pseudo-science again right?".... and as far as I can see, yes, basically it is. 1. Logical bits are not thermodynamic bits, so a thermodynamic analysis of the brain can only be compared to a thermodynamic analysis of CPUs (if one wishes to compare at all) 2. Every phrase which begins "suppose, assume, conjecture" introduces fatal assuptions into the whole project, extremely few are defensible. The idea that a single neurone is "computing" whilst its "communication" with others is non-computational in a logical sense is clearly false. One can make a thermodynmic distinction between energy "of the neurone" and "of thier 'communication'" this distinction has no relevance to a computational-logical model of the brain as a computational system. 3. The frequent reference to thermodynamic limits of "computation" (in a logical sense) as a baseline for comparison with (abitary) parts of the brain, is meaningless. The thermodynamic efficiency of the brain is interesting only insofar as any possible logical model of the brain seems to imply vastly more "computational resources". And vastly more thermodynamic resources than CPUs have. Physical limits on theoretical computation pertain, if they are even themsevles coherent (which is disputed), to the absolute minimal possible "piece" of reality, not even, i'd say, to any measurable phenomenon. As soon as a system has to engage in measurement, i'd say it would be millions+ times less "efficient" than 'physical limit's would suggest. One "trick" to see through the pseudoscience of computational neurobiology is simply to apply its methods to actual CPUs and computers. With the above assumptions, this paper would conclude that only one operation in a transistor is "computing" anything, the energy required to do that is "the energy of computation"... and the rest of the energy used across CPU(-RAM-etc.) was "merely communication". As-if all algorithms were logically specified as a purely parallel series of switch-flips. No, algorithms (in a computation-logic) sense have nothing to do with switch flips. And their implementation on digital computers requires many, serial and parallel and "thermodynamic communication" between them to perform the computation in question.
- mjburgess 4y ago
  Adding an additional comment here, as I'm thinking to myself about how I'd clarify the issue further. Suppose we write a program in `C` which requires 32 bit array, sets an input state, operates on the array, and produces an output state. This program is the logical model, here requiring say 32bits and 10 operations/bit average. So we have 320 logical changes to the array (which btw, wont be purely parallel or serial). Suppose the most efficient CPU we have requires E_thermo energy for this whole process (setting the 32bit input, operating, reading the output). Now, using somewhat disputed ideas about physical limits, E_thermo say implies we've used 320,000 'bits' thermodynamically, ie., somehow the CPU has produced an equivalent of 320,000 "energetic changes". So the CPU has an efficiency, in some suspicious sense, of 100,000 thermo-bits / logical-bit. For each single on-average change to the array we see in the `C program`, we theoretically measure an equivalent heat of 100,000 changes. Now, how efficient is the brain at computation? Well since we have no logical model of the computation its performing, there's absolutely no way of answering that question. So why do people say it's efficient? Well because any apparently equivalent algorithm we come up with (in, eg., `C`) to do even basic things that the brain does, requires vast amounts more energy. Eg., processing images on a GPU requires, say 300 W, and our whole brain uses (a claimed) 20 W. I think the resolution to this problem is that the brain(-environment-body) system implements computation in a radically different manner than anything like a CPU, GPU, etc. I suspect that computation happens at every scale: molecular, sub-cellular, cellular, neuronal, inter-neuronal, nervous-system, body, body-environment, etc. And across the whole body, and using energy in the environment to maintain state. However, I find the use of the term "computer" and "computational" mostly just productive of pseudoscience. It is a basically meaningless term that only produces confusion. The only useful area where it helps is when handling logical models of algorithms -- models largely absent from the whole of empirical and theoretical science.
apienx 4y ago
The PNAS editors involved in this clickbait-y title need some introspection. You can be rigorous and generalist.
deleted 4y ago
[deleted]
max_entropy 4y ago
How can computation be separated from communication?
- SemanticStrengh 4y ago
  Asking the real questions. Axons are connected to the neuron and exchange a lot of chemicals between them. For example, the mitochondria in axons go back and fort to the neuron IIRC. It's not very googlable but neurons have a memory of how likely they are to trigger an action potential, based on past stimuluses/timings. No one knows where this memory is encoded. And is there one action potential per axon? If so then it would be more logical for this memory (and computation which leads to mutation to the memory) to be stored in the axon (if).
- tombh 4y ago
  I think Noam Chomsky even believes that human's language ability was selected for to aid computation not communication. Therefore in humans: communication IS computation.
- lambdatronics 4y ago
  That's a really profound question that deserves serious research, IMO. Communication is easy enough to define using Shannon's theory of information. I would argue that computation is subset of communication: signals go in one end of a 'processor' & come out the other (so the processor acts as a 'channel' in communication theory), but the data comes out modified in some non-trivial way, which distinguishes computation/processing of information from mere replication of the signals at the output. A trivial modification would be something like (X,Y) -> (Y,X), just re-ordering cables. A non-trivial modification would be something like (X,Y) -> (X XOR Y, X AND Y). Both of these 'channels' have 2 bits of capacity, but intuitively only the second one is actually doing any information processing/computing. I haven't found a satisfactory way to formalize my intuition about what is 'non-trivial' and to quantify how much 'processing' is being done by a general system, in such a way that one could meaningfully compare a brain and a CPU, for example. I think it's a great research question.
  - 4y ago
SemanticStrengh 4y ago
Also cognition only consume 0.2 watts
ineedasername 4y ago
I dunt ned thet metch too comkate, mebe 2x. Mkay, 3x.
SemanticStrengh 4y ago
People should review how much this study is coherent of not with this one https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.full https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1....
- photochemsyn 4y ago
  Really, this whole field seems like woo. It seems safe to conclude that neurons are not taking in sensory data, converting that data to numbers, then running little discrete algorithms to decide whether the external stimuli can be safely ignored or not, using other numbers stored in their internal registers to modify that decision. The von Neuman cycle of fetch-decode-execute is not a useful model for coordinated neuronal activity, and logical bit operations have little relevance to biological neuronal activity. If people want to take a look at more reliable science on this area, this is not a bad place to start. In particular look at the 141 citations. Central command neurons of the sympathetic nervous system: basis of the fight-or-flight response (Science 1995) https://pubmed.ncbi.nlm.nih.gov/7570024/ https://pubmed.ncbi.nlm.nih.gov/7570024/ This is important and interesting science, and has relevance to understanding and treating things like anxiety disorders (probably related to hypersensitivity to external stimuli), but there's no evidence that any 'computational model' has much relevance. These systems really appear to be analog in nature, not at all digital. Also, the lead statement in that paper, i.e. Darwinian evolution tends to produce energy-efficient outcomes, may sound good but in real cases it seems to vary a lot. While a flight-or-fight response is likely under constant positive selection (and hence energy-efficient), think about animal mating behavior. Are the endless mating rituals, breeding displays, etc. at all 'energy efficient'? Large amounts of brain energy are devoted to such activities in all species. Certainly there are evolutionary arguments for this, but energy efficiency is not the main issue. Perhaps, one could claim that each individual neuron is something like a CPU, in that it has state, something like an internal memory, but it's a rather vague relationship. Maybe a network of biological neurons is something like a network of connected CPUs, but it seems doubtful such CPUs rely on Arithmetic and Logic Units or bitwise operations.
anonymousDan 4y ago
It reminds me a bit of the field of wireless sensor networks from back in the day, where the main challenge was that wireless communication was typically much more energy intensive than local computation requirements and so there was an incentive to perform additional computation locally that might reduce communication (e.g. to filter or aggregate data).
colordrops 4y ago
Sounds like AWS.
noduerme 4y ago
This feels like it wad generated by GPT-3. Am I going nuts? I just came to HN after falling into a whirlpool of "Best Coffee Makers for 2022" SEO spam sites (waste of a Sunday morning), and noting how each AI-written "review" followed the same introductory pattern of chitchat. And then I come here and find a paper that sounds like it's by the same AI author. >> Darwinian evolution tends to produce energy-efficient outcomes. On the other hand, energy limits computation, be it neural and probabilistic or digital and logical. Taking a particular energy-efficient viewpoint, Someone tell me that's not GPT. Compare with this from a coffeemaker fake review site: >> The more features something has, the more you can make use of it, such is the principle rule of products and services alike. This rule is why companies try to compete with each other on their best products. >> Hence, the brand Imusa is no exception to this, as they make an electric mocha maker that is loaded with features. To unpack all of the features, we must start with the build quality and work all the way down to the specifics. [edited to add comparison]
- echelon 4y ago
  Your direct quote sounds like any scientific paper.
- svnt 4y ago
  It’s the loose use of “on the other hand” when the two constraints are not clearly first presented as being in opposition. I am frankly not sure that either of those are GPT-generated —- perhaps they can be read instead as illustrative of word-count driven human egotism and inconsistent writing.
  - noduerme 4y ago
    I think you nailed it. I think GPT has a poker tell. The tell is beginning the second sentence or paragraph with "on the other hand", followed by a statement which doesn't directly relate to or contradict the opener. When real people say "on the other hand", it usually precedes something that's in direct opposition to whatever logical point they began with. [edit] FWIW, this behavior is startlingly similar to the way storefront psychics conduct a "cold read". They present two views which are both agreeable but could be considered contradictory, and wait to see which one their mark gravitates to. It's likely we've only programmed AIs that are modeled to trip people's emotional wires. Basically Eliza 3.0.
rileyphone 4y ago
This is in line with [0], which argues that modularity in small-world networks produced by evolutionary processes is largely a result of connection costs. It’s unfortunate that science here has such a limited empirical footing given the importance of the knowledge, but that doesn’t make it wrong, just circumspect. 0. https://royalsocietypublishing.org/doi/10.1098/rspb.2012.2863 https://royalsocietypublishing.org/doi/10.1098/rspb.2012.286...

151 points by ofou 4y ago

anotherhue 4y ago

intra-neuronal communication, not 'talking to people' (tiring as that may be).

calculated 4y ago
Thank you for this specification as the OP hasn't put it in the title. Before reading your comment I thought the article is about human interaction.
- devonallie 4y ago
  Maybe it’s because I’ve been spending too much time thinking about packets, but I thought it was “digital” communication.
xwdv 4y ago
It’s crazy how I was willing to believe that this is what the article was talking about without even opening it. Authors should be more specific, a lot of misinformation could be spread through misleading titles.
- nanofortnight 4y ago
  "Communication in the human cortex" seems pretty clear. The abstract is there for an extended summary if one doesn't wish to read the whole article. One cannot possibly understand the nuances of something without reading it; I do not feel knowledge gained through reading titles of articles is usable knowledge. Perhaps one should consider changing the habits of the reader rather than the author?
  - xwdv 4y ago
    A lot of that nuance would be quickly forgotten, but the info from a good title could last a long time.
layer8 4y ago
I believe you mean inter-neural communication? But, in consequence, by “computation” they must mean intra-neural computation? Because otherwise (i.e. for inter-neural computation) the “communication” would be an integral part of the computation. Or do they really mean that most of the energy cost of neural computation must be attributed to inter-neural communication?

mjburgess 4y ago

I have friends who are non-science PhDs and absolutely believe anything published in a science journal must be absolutely sound. Before I started reading I thought, "alas, its going to be pseudo-science again right?".... and as far as I can see, yes, basically it is. 1. Logical bits are not thermodynamic bits, so a thermodynamic analysis of the brain can only be compared to a thermodynamic analysis of CPUs (if one wishes to compare at all) 2. Every phrase which begins "suppose, assume, conjecture" introduces fatal assuptions into the whole project, extremely few are defensible. The idea that a single neurone is "computing" whilst its "communication" with others is non-computational in a logical sense is clearly false. One can make a thermodynmic distinction between energy "of the neurone" and "of thier 'communication'" this distinction has no relevance to a computational-logical model of the brain as a computational system. 3. The frequent reference to thermodynamic limits of "computation" (in a logical sense) as a baseline for comparison with (abitary) parts of the brain, is meaningless. The thermodynamic efficiency of the brain is interesting only insofar as any possible logical model of the brain seems to imply vastly more "computational resources". And vastly more thermodynamic resources than CPUs have. Physical limits on theoretical computation pertain, if they are even themsevles coherent (which is disputed), to the absolute minimal possible "piece" of reality, not even, i'd say, to any measurable phenomenon. As soon as a system has to engage in measurement, i'd say it would be millions+ times less "efficient" than 'physical limit's would suggest. One "trick" to see through the pseudoscience of computational neurobiology is simply to apply its methods to actual CPUs and computers. With the above assumptions, this paper would conclude that only one operation in a transistor is "computing" anything, the energy required to do that is "the energy of computation"... and the rest of the energy used across CPU(-RAM-etc.) was "merely communication". As-if all algorithms were logically specified as a purely parallel series of switch-flips. No, algorithms (in a computation-logic) sense have nothing to do with switch flips. And their implementation on digital computers requires many, serial and parallel and "thermodynamic communication" between them to perform the computation in question.

mjburgess 4y ago
Adding an additional comment here, as I'm thinking to myself about how I'd clarify the issue further. Suppose we write a program in `C` which requires 32 bit array, sets an input state, operates on the array, and produces an output state. This program is the logical model, here requiring say 32bits and 10 operations/bit average. So we have 320 logical changes to the array (which btw, wont be purely parallel or serial). Suppose the most efficient CPU we have requires E_thermo energy for this whole process (setting the 32bit input, operating, reading the output). Now, using somewhat disputed ideas about physical limits, E_thermo say implies we've used 320,000 'bits' thermodynamically, ie., somehow the CPU has produced an equivalent of 320,000 "energetic changes". So the CPU has an efficiency, in some suspicious sense, of 100,000 thermo-bits / logical-bit. For each single on-average change to the array we see in the `C program`, we theoretically measure an equivalent heat of 100,000 changes. Now, how efficient is the brain at computation? Well since we have no logical model of the computation its performing, there's absolutely no way of answering that question. So why do people say it's efficient? Well because any apparently equivalent algorithm we come up with (in, eg., `C`) to do even basic things that the brain does, requires vast amounts more energy. Eg., processing images on a GPU requires, say 300 W, and our whole brain uses (a claimed) 20 W. I think the resolution to this problem is that the brain(-environment-body) system implements computation in a radically different manner than anything like a CPU, GPU, etc. I suspect that computation happens at every scale: molecular, sub-cellular, cellular, neuronal, inter-neuronal, nervous-system, body, body-environment, etc. And across the whole body, and using energy in the environment to maintain state. However, I find the use of the term "computer" and "computational" mostly just productive of pseudoscience. It is a basically meaningless term that only produces confusion. The only useful area where it helps is when handling logical models of algorithms -- models largely absent from the whole of empirical and theoretical science.

apienx 4y ago

The PNAS editors involved in this clickbait-y title need some introspection. You can be rigorous and generalist.

deleted 4y ago

[deleted]

max_entropy 4y ago

How can computation be separated from communication?

SemanticStrengh 4y ago
Asking the real questions. Axons are connected to the neuron and exchange a lot of chemicals between them. For example, the mitochondria in axons go back and fort to the neuron IIRC. It's not very googlable but neurons have a memory of how likely they are to trigger an action potential, based on past stimuluses/timings. No one knows where this memory is encoded. And is there one action potential per axon? If so then it would be more logical for this memory (and computation which leads to mutation to the memory) to be stored in the axon (if).
tombh 4y ago
I think Noam Chomsky even believes that human's language ability was selected for to aid computation not communication. Therefore in humans: communication IS computation.
lambdatronics 4y ago
That's a really profound question that deserves serious research, IMO. Communication is easy enough to define using Shannon's theory of information. I would argue that computation is subset of communication: signals go in one end of a 'processor' & come out the other (so the processor acts as a 'channel' in communication theory), but the data comes out modified in some non-trivial way, which distinguishes computation/processing of information from mere replication of the signals at the output. A trivial modification would be something like (X,Y) -> (Y,X), just re-ordering cables. A non-trivial modification would be something like (X,Y) -> (X XOR Y, X AND Y). Both of these 'channels' have 2 bits of capacity, but intuitively only the second one is actually doing any information processing/computing. I haven't found a satisfactory way to formalize my intuition about what is 'non-trivial' and to quantify how much 'processing' is being done by a general system, in such a way that one could meaningfully compare a brain and a CPU, for example. I think it's a great research question.
- 4y ago

SemanticStrengh 4y ago

Also cognition only consume 0.2 watts

ineedasername 4y ago

I dunt ned thet metch too comkate, mebe 2x. Mkay, 3x.

SemanticStrengh 4y ago

People should review how much this study is coherent of not with this one https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.full https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1....

photochemsyn 4y ago
Really, this whole field seems like woo. It seems safe to conclude that neurons are not taking in sensory data, converting that data to numbers, then running little discrete algorithms to decide whether the external stimuli can be safely ignored or not, using other numbers stored in their internal registers to modify that decision. The von Neuman cycle of fetch-decode-execute is not a useful model for coordinated neuronal activity, and logical bit operations have little relevance to biological neuronal activity. If people want to take a look at more reliable science on this area, this is not a bad place to start. In particular look at the 141 citations. Central command neurons of the sympathetic nervous system: basis of the fight-or-flight response (Science 1995) https://pubmed.ncbi.nlm.nih.gov/7570024/ https://pubmed.ncbi.nlm.nih.gov/7570024/ This is important and interesting science, and has relevance to understanding and treating things like anxiety disorders (probably related to hypersensitivity to external stimuli), but there's no evidence that any 'computational model' has much relevance. These systems really appear to be analog in nature, not at all digital. Also, the lead statement in that paper, i.e. Darwinian evolution tends to produce energy-efficient outcomes, may sound good but in real cases it seems to vary a lot. While a flight-or-fight response is likely under constant positive selection (and hence energy-efficient), think about animal mating behavior. Are the endless mating rituals, breeding displays, etc. at all 'energy efficient'? Large amounts of brain energy are devoted to such activities in all species. Certainly there are evolutionary arguments for this, but energy efficiency is not the main issue. Perhaps, one could claim that each individual neuron is something like a CPU, in that it has state, something like an internal memory, but it's a rather vague relationship. Maybe a network of biological neurons is something like a network of connected CPUs, but it seems doubtful such CPUs rely on Arithmetic and Logic Units or bitwise operations.

anonymousDan 4y ago

It reminds me a bit of the field of wireless sensor networks from back in the day, where the main challenge was that wireless communication was typically much more energy intensive than local computation requirements and so there was an incentive to perform additional computation locally that might reduce communication (e.g. to filter or aggregate data).

colordrops 4y ago

Sounds like AWS.

noduerme 4y ago

This feels like it wad generated by GPT-3. Am I going nuts? I just came to HN after falling into a whirlpool of "Best Coffee Makers for 2022" SEO spam sites (waste of a Sunday morning), and noting how each AI-written "review" followed the same introductory pattern of chitchat. And then I come here and find a paper that sounds like it's by the same AI author. >> Darwinian evolution tends to produce energy-efficient outcomes. On the other hand, energy limits computation, be it neural and probabilistic or digital and logical. Taking a particular energy-efficient viewpoint, Someone tell me that's not GPT. Compare with this from a coffeemaker fake review site: >> The more features something has, the more you can make use of it, such is the principle rule of products and services alike. This rule is why companies try to compete with each other on their best products. >> Hence, the brand Imusa is no exception to this, as they make an electric mocha maker that is loaded with features. To unpack all of the features, we must start with the build quality and work all the way down to the specifics. [edited to add comparison]

echelon 4y ago
Your direct quote sounds like any scientific paper.
svnt 4y ago
It’s the loose use of “on the other hand” when the two constraints are not clearly first presented as being in opposition. I am frankly not sure that either of those are GPT-generated —- perhaps they can be read instead as illustrative of word-count driven human egotism and inconsistent writing.
- noduerme 4y ago
  I think you nailed it. I think GPT has a poker tell. The tell is beginning the second sentence or paragraph with "on the other hand", followed by a statement which doesn't directly relate to or contradict the opener. When real people say "on the other hand", it usually precedes something that's in direct opposition to whatever logical point they began with. [edit] FWIW, this behavior is startlingly similar to the way storefront psychics conduct a "cold read". They present two views which are both agreeable but could be considered contradictory, and wait to see which one their mark gravitates to. It's likely we've only programmed AIs that are modeled to trip people's emotional wires. Basically Eliza 3.0.

rileyphone 4y ago

This is in line with [0], which argues that modularity in small-world networks produced by evolutionary processes is largely a result of connection costs. It’s unfortunate that science here has such a limited empirical footing given the importance of the knowledge, but that doesn’t make it wrong, just circumspect. 0. https://royalsocietypublishing.org/doi/10.1098/rspb.2012.2863 https://royalsocietypublishing.org/doi/10.1098/rspb.2012.286...