6 ms·

I have friends who are non-science PhDs and absolutely believe anything published in a science journal must be absolutely sound. Before I started reading I tho

I have friends who are non-science PhDs and absolutely believe anything published in a science journal must be absolutely sound. Before I started reading I thought, "alas, its going to be pseudo-science again right?".... and as far as I can see, yes, basically it is. 1. Logical bits are not thermodynamic bits, so a thermodynamic analysis of the brain can only be compared to a thermodynamic analysis of CPUs (if one wishes to compare at all) 2. Every phrase which begins "suppose, assume, conjecture" introduces fatal assuptions into the whole project, extremely few are defensible. The idea that a single neurone is "computing" whilst its "communication" with others is non-computational in a logical sense is clearly false. One can make a thermodynmic distinction between energy "of the neurone" and "of thier 'communication'" this distinction has no relevance to a computational-logical model of the brain as a computational system. 3. The frequent reference to thermodynamic limits of "computation" (in a logical sense) as a baseline for comparison with (abitary) parts of the brain, is meaningless. The thermodynamic efficiency of the brain is interesting only insofar as any possible logical model of the brain seems to imply vastly more "computational resources". And vastly more thermodynamic resources than CPUs have. Physical limits on theoretical computation pertain, if they are even themsevles coherent (which is disputed), to the absolute minimal possible "piece" of reality, not even, i'd say, to any measurable phenomenon. As soon as a system has to engage in measurement, i'd say it would be millions+ times less "efficient" than 'physical limit's would suggest. One "trick" to see through the pseudoscience of computational neurobiology is simply to apply its methods to actual CPUs and computers. With the above assumptions, this paper would conclude that only one operation in a transistor is "computing" anything, the energy required to do that is "the energy of computation"... and the rest of the energy used across CPU(-RAM-etc.) was "merely communication". As-if all algorithms were logically specified as a purely parallel series of switch-flips. No, algorithms (in a computation-logic) sense have nothing to do with switch flips. And their implementation on digital computers requires many, serial and parallel and "thermodynamic communication" between them to perform the computation in question.

mjburgess 4y ago
Adding an additional comment here, as I'm thinking to myself about how I'd clarify the issue further. Suppose we write a program in `C` which requires 32 bit array, sets an input state, operates on the array, and produces an output state. This program is the logical model, here requiring say 32bits and 10 operations/bit average. So we have 320 logical changes to the array (which btw, wont be purely parallel or serial). Suppose the most efficient CPU we have requires E_thermo energy for this whole process (setting the 32bit input, operating, reading the output). Now, using somewhat disputed ideas about physical limits, E_thermo say implies we've used 320,000 'bits' thermodynamically, ie., somehow the CPU has produced an equivalent of 320,000 "energetic changes". So the CPU has an efficiency, in some suspicious sense, of 100,000 thermo-bits / logical-bit. For each single on-average change to the array we see in the `C program`, we theoretically measure an equivalent heat of 100,000 changes. Now, how efficient is the brain at computation? Well since we have no logical model of the computation its performing, there's absolutely no way of answering that question. So why do people say it's efficient? Well because any apparently equivalent algorithm we come up with (in, eg., `C`) to do even basic things that the brain does, requires vast amounts more energy. Eg., processing images on a GPU requires, say 300 W, and our whole brain uses (a claimed) 20 W. I think the resolution to this problem is that the brain(-environment-body) system implements computation in a radically different manner than anything like a CPU, GPU, etc. I suspect that computation happens at every scale: molecular, sub-cellular, cellular, neuronal, inter-neuronal, nervous-system, body, body-environment, etc. And across the whole body, and using energy in the environment to maintain state. However, I find the use of the term "computer" and "computational" mostly just productive of pseudoscience. It is a basically meaningless term that only produces confusion. The only useful area where it helps is when handling logical models of algorithms -- models largely absent from the whole of empirical and theoretical science.
- rad88 4y ago
  What did you mean by "computation" then? Honestly curious, because it seems you could also say a digital computer does it at all scales: semiconductor junctions, capacitors, clocks, their multipliers, transistors, up through logic gates, blocks, buses, ..., and all the way out to the fans, power supply, etc?
  - mjburgess 4y ago
    I mean that if "intelligence (etc.)" in the relevant sense can be described by an algorithm, its implementation will not just depend on properties neurones have as neurones. It'll depend on properties at various scales. This isnt quite true for digital computers, in the sense that the only property which implements the algorithm is the electrical switching state... that this state depends on, eg., silicon isnt quite the same as silicon-properties the ones "doing the work". This is part of the trouble talking about "computation" at all, which is a nearly empty term in my view. Consider an example. A "baking a cake" algorithm can be implemented by a person or a factory, if specified correctly. In one case, the organic properties of a person appear "essential to the implementation", but they arent. But there are some algorithms which do require organic properties (eg., if we consider the behaviour of a cell wall an algorithm) to implement. The question is in what sense are the implementation properties "doing the computational work". Any electrical system which provides switching states is "doing the work in the same way", ie., there are certain "macro-properties" which do the work, regardless of their micro-property dependencies. I think in the case of intelligence (the brain-body, etc.) the properties "doing the computational work" arent the macro-properties of neuronal interaction. They're properties at various scales (subcelluar protein behaviours; celluar interation; neuronal communication; bodily organization; environmental driving; etc.). In particular, i think it's the self-adapting self-organizing properties of certain organic systems that allow them to implement "the intelligence algorithm" -- this algorithm, if it can ever be specified, I do not think will be implemented only by macroscopic neurone firing -- i'd say that'll be a very minor part of its implementation. I think most of it will come from self-organization properties from the subcell to the body.
    - 4y ago
dav_Oz 4y ago
Your very valid points are - in my reading - addressed in this paper, albeit in a milder and more forgiving form, in the Results section under A Baseline for Maximally Efficient Computation. A simplistic model relates physics to neuroscience. Especially: That is, physics looks at each computational element only as a solitary individual, performing but a single operation. There is no consideration that each neuron participates in a large network or even that a logical gate must communicate its inference in a digital computer in a timely manner. Unlike idealized physics, Nature cannot afford to ignore the energy requirements arising from communication and time constraints that are fundamental network considerations (43) and fundamental to survival itself (especially time) (18, 19). I wouldn't call it outright pseudo-science as the authors are trying to impose some limited working framework with very technical meanings of "communication" and "computation" in this context. I too, find this "physics"/"cs" excursion very basic and questionable which kind of points to the purpose of that paper: demonstrating an exercise. And as a general symptom the difficulty to overcome the shallow waters of interdisciplinary fields/approaches. [From my own experience in talking with biologists only after awhile I begin to appreciate the deep complexities of biological/biochemical systems, only to - after some time has passed - forget those subtleties again. I'm so used to simplify from my physics background that it is actually quite hard to recognize important distinctions in different (more complex) fields which can lead to vastly different mathematical models.]
- mjburgess 4y ago
  If this were one paper in isolation, I'd call it an honest attempt at good science gone-wrong by a lack of consultation with theoretical computer scientists and specialists in thermodynamics. However, I think this is just *the entire field* -- which is then different. I think a whole field is pseudoscience when it systematically engages in the same games. I think likewise of much fMRI work,... all the way to basically the whole of nutritional science, etc. If the premises of your project can be pretty quickly falsified by relevant domain experts, and you build a whole field out of it, you slip into my category of "pseudoscience". This paper explicitly tries to prescribe to engineers (!!!) where their "focus" should be, having absolutely no warrant to do so. A trivial application of the same analysis to CPUs would provide the relevant baseline comparison the paper should have made. With this analysis, the key results of the paper would be exposed as useless. I have little sympathy for all this now: it's a research hype cycle craze to apply random bits of computer science to random bits of science in the most hairbrained manner.
gnramires 4y ago
I think you're missing reading non-hard sciences. In science every analysis that clear states its assumptions and models is valid. You don't have to derive everything from axioms and physical laws. You can make assumptions and simplifying models and operate within that, as long as you clearly state your assumptions that may or may not hold -- that's fine. This is common in engineering papers. > Logical bits are not thermodynamic bits They are lower bounded by thermodynamic bits? (in the sense of energy for instance) The thermodynamic bits are exactly that: information (although it seems information in thermodynamic theory is still not perfectly well understood).
- mjburgess 4y ago
  They arent lower-bounded by thermo bits --- because one can specify algorithms which require no thermodynamic work to implement. Logical bits and thermo bits are related by contingent facts of implementation. One has first to specify an algorithm (defined in terms of a computational model), then it's an open question as to what-and-how it'll be implemented. It's also not at all clear that the physics terms "entropy, information, bits, etc." have anything to do with their computational "equivalents". Only by fairly strained thought experiments do we get alleged connections. Even these thought experiments only provide extremely limited translation of these terms between domains. "Information" in a "logical" sense is a radically different think than in a "thermodynamic sense"... for example, the former has an obvious observer-independent definition, the latter does not. The whole game of trying to bridge these notions without specifying implementation relations (etc.) is largely the new form of that transhumanism-craze: the respectable ideological space of delusional techno-utopian hopes.
  - mordechai9000 4y ago
    > one can specify algorithms which require no thermodynamic work to implement Can you give an example? To my limited understanding, performing work without expending energy sounds like perpetual motion.
    - mjburgess 4y ago
titzer 4y ago
> With the above assumptions, this paper would conclude that only one operation in a transistor is "computing" anything Why? A transistor is just an arbitrary boundary. You could as well draw a box around any combinatorial circuit and just call it a "computational element". It has inputs and outputs and a switching energy for each unit of computation it does. This is just the process of breaking computation up into abstractions--"boxes". Clearly we must do this, otherwise we'd have to reason about individual electrons.
- svnt 4y ago
  Electrons themselves being just easily-measured boundaries around littler bits we’d be forced to acknowledge that our entire modeled reality is constructed.
  - xpe 4y ago
    The abstractions are constructed, yes. Probability suggests the same reality is there across a wide variety of experience -- or at least the experience that has been experimented on and discussed by the scientific community.
kingkawn 4y ago
Well said. I’d only add that neurons are part of an exponentially higher dimensional calculation process than binary, taking input from a huge number of chemicals, proteins, electrical signals, insulation variability, and who knows what other undiscovered dynamics that are at play. There is no 1:1 comparison between computers and the brain anymore than there was between aqueducts and the brain.

mjburgess 4y ago

Adding an additional comment here, as I'm thinking to myself about how I'd clarify the issue further. Suppose we write a program in `C` which requires 32 bit array, sets an input state, operates on the array, and produces an output state. This program is the logical model, here requiring say 32bits and 10 operations/bit average. So we have 320 logical changes to the array (which btw, wont be purely parallel or serial). Suppose the most efficient CPU we have requires E_thermo energy for this whole process (setting the 32bit input, operating, reading the output). Now, using somewhat disputed ideas about physical limits, E_thermo say implies we've used 320,000 'bits' thermodynamically, ie., somehow the CPU has produced an equivalent of 320,000 "energetic changes". So the CPU has an efficiency, in some suspicious sense, of 100,000 thermo-bits / logical-bit. For each single on-average change to the array we see in the `C program`, we theoretically measure an equivalent heat of 100,000 changes. Now, how efficient is the brain at computation? Well since we have no logical model of the computation its performing, there's absolutely no way of answering that question. So why do people say it's efficient? Well because any apparently equivalent algorithm we come up with (in, eg., `C`) to do even basic things that the brain does, requires vast amounts more energy. Eg., processing images on a GPU requires, say 300 W, and our whole brain uses (a claimed) 20 W. I think the resolution to this problem is that the brain(-environment-body) system implements computation in a radically different manner than anything like a CPU, GPU, etc. I suspect that computation happens at every scale: molecular, sub-cellular, cellular, neuronal, inter-neuronal, nervous-system, body, body-environment, etc. And across the whole body, and using energy in the environment to maintain state. However, I find the use of the term "computer" and "computational" mostly just productive of pseudoscience. It is a basically meaningless term that only produces confusion. The only useful area where it helps is when handling logical models of algorithms -- models largely absent from the whole of empirical and theoretical science.

rad88 4y ago
What did you mean by "computation" then? Honestly curious, because it seems you could also say a digital computer does it at all scales: semiconductor junctions, capacitors, clocks, their multipliers, transistors, up through logic gates, blocks, buses, ..., and all the way out to the fans, power supply, etc?
- mjburgess 4y ago
  I mean that if "intelligence (etc.)" in the relevant sense can be described by an algorithm, its implementation will not just depend on properties neurones have as neurones. It'll depend on properties at various scales. This isnt quite true for digital computers, in the sense that the only property which implements the algorithm is the electrical switching state... that this state depends on, eg., silicon isnt quite the same as silicon-properties the ones "doing the work". This is part of the trouble talking about "computation" at all, which is a nearly empty term in my view. Consider an example. A "baking a cake" algorithm can be implemented by a person or a factory, if specified correctly. In one case, the organic properties of a person appear "essential to the implementation", but they arent. But there are some algorithms which do require organic properties (eg., if we consider the behaviour of a cell wall an algorithm) to implement. The question is in what sense are the implementation properties "doing the computational work". Any electrical system which provides switching states is "doing the work in the same way", ie., there are certain "macro-properties" which do the work, regardless of their micro-property dependencies. I think in the case of intelligence (the brain-body, etc.) the properties "doing the computational work" arent the macro-properties of neuronal interaction. They're properties at various scales (subcelluar protein behaviours; celluar interation; neuronal communication; bodily organization; environmental driving; etc.). In particular, i think it's the self-adapting self-organizing properties of certain organic systems that allow them to implement "the intelligence algorithm" -- this algorithm, if it can ever be specified, I do not think will be implemented only by macroscopic neurone firing -- i'd say that'll be a very minor part of its implementation. I think most of it will come from self-organization properties from the subcell to the body.
  - 4y ago

dav_Oz 4y ago

Your very valid points are - in my reading - addressed in this paper, albeit in a milder and more forgiving form, in the Results section under A Baseline for Maximally Efficient Computation. A simplistic model relates physics to neuroscience. Especially: That is, physics looks at each computational element only as a solitary individual, performing but a single operation. There is no consideration that each neuron participates in a large network or even that a logical gate must communicate its inference in a digital computer in a timely manner. Unlike idealized physics, Nature cannot afford to ignore the energy requirements arising from communication and time constraints that are fundamental network considerations (43) and fundamental to survival itself (especially time) (18, 19). I wouldn't call it outright pseudo-science as the authors are trying to impose some limited working framework with very technical meanings of "communication" and "computation" in this context. I too, find this "physics"/"cs" excursion very basic and questionable which kind of points to the purpose of that paper: demonstrating an exercise. And as a general symptom the difficulty to overcome the shallow waters of interdisciplinary fields/approaches. [From my own experience in talking with biologists only after awhile I begin to appreciate the deep complexities of biological/biochemical systems, only to - after some time has passed - forget those subtleties again. I'm so used to simplify from my physics background that it is actually quite hard to recognize important distinctions in different (more complex) fields which can lead to vastly different mathematical models.]

mjburgess 4y ago
If this were one paper in isolation, I'd call it an honest attempt at good science gone-wrong by a lack of consultation with theoretical computer scientists and specialists in thermodynamics. However, I think this is just *the entire field* -- which is then different. I think a whole field is pseudoscience when it systematically engages in the same games. I think likewise of much fMRI work,... all the way to basically the whole of nutritional science, etc. If the premises of your project can be pretty quickly falsified by relevant domain experts, and you build a whole field out of it, you slip into my category of "pseudoscience". This paper explicitly tries to prescribe to engineers (!!!) where their "focus" should be, having absolutely no warrant to do so. A trivial application of the same analysis to CPUs would provide the relevant baseline comparison the paper should have made. With this analysis, the key results of the paper would be exposed as useless. I have little sympathy for all this now: it's a research hype cycle craze to apply random bits of computer science to random bits of science in the most hairbrained manner.

gnramires 4y ago

I think you're missing reading non-hard sciences. In science every analysis that clear states its assumptions and models is valid. You don't have to derive everything from axioms and physical laws. You can make assumptions and simplifying models and operate within that, as long as you clearly state your assumptions that may or may not hold -- that's fine. This is common in engineering papers. > Logical bits are not thermodynamic bits They are lower bounded by thermodynamic bits? (in the sense of energy for instance) The thermodynamic bits are exactly that: information (although it seems information in thermodynamic theory is still not perfectly well understood).

mjburgess 4y ago
They arent lower-bounded by thermo bits --- because one can specify algorithms which require no thermodynamic work to implement. Logical bits and thermo bits are related by contingent facts of implementation. One has first to specify an algorithm (defined in terms of a computational model), then it's an open question as to what-and-how it'll be implemented. It's also not at all clear that the physics terms "entropy, information, bits, etc." have anything to do with their computational "equivalents". Only by fairly strained thought experiments do we get alleged connections. Even these thought experiments only provide extremely limited translation of these terms between domains. "Information" in a "logical" sense is a radically different think than in a "thermodynamic sense"... for example, the former has an obvious observer-independent definition, the latter does not. The whole game of trying to bridge these notions without specifying implementation relations (etc.) is largely the new form of that transhumanism-craze: the respectable ideological space of delusional techno-utopian hopes.
- mordechai9000 4y ago
  > one can specify algorithms which require no thermodynamic work to implement Can you give an example? To my limited understanding, performing work without expending energy sounds like perpetual motion.
  - mjburgess 4y ago

titzer 4y ago

> With the above assumptions, this paper would conclude that only one operation in a transistor is "computing" anything Why? A transistor is just an arbitrary boundary. You could as well draw a box around any combinatorial circuit and just call it a "computational element". It has inputs and outputs and a switching energy for each unit of computation it does. This is just the process of breaking computation up into abstractions--"boxes". Clearly we must do this, otherwise we'd have to reason about individual electrons.

svnt 4y ago
Electrons themselves being just easily-measured boundaries around littler bits we’d be forced to acknowledge that our entire modeled reality is constructed.
- xpe 4y ago
  The abstractions are constructed, yes. Probability suggests the same reality is there across a wide variety of experience -- or at least the experience that has been experimented on and discussed by the scientific community.

kingkawn 4y ago

Well said. I’d only add that neurons are part of an exponentially higher dimensional calculation process than binary, taking input from a huge number of chemicals, proteins, electrical signals, insulation variability, and who knows what other undiscovered dynamics that are at play. There is no 1:1 comparison between computers and the brain anymore than there was between aqueducts and the brain.