5 ms·

The complexity of the human body surely weighs in at over 1 GB. I think of DNA analogously to the rules of cellular automata. The entropy of the rules is much

The complexity of the human body surely weighs in at over 1 GB. I think of DNA analogously to the rules of cellular automata. The entropy of the rules is much less than the entropy of the dynamical system the rules describe. The body is filled with innate knowledge. The organs all know what to do. The immune system learns to detect intruders (without synapses). Even a single cell organism is capable of complex and fluid goal-oriented behavior, as Michael Levine attests. I think the assumption that all knowledge exists in the brain, and all knowledge in the brain is encoded by neuronal weights, is probably too simplistic. Regarding language and vision, I think the cognitive scientists are right: it is better to view these as organs or “modules” suited to a function. Damage Broca’s area and you get Broca’s aphasia. Damage your lung and you get trouble breathing. Neither of these looks like the result of statistical learning from randomly initialized parameters.

ACCount37 1y ago
Damage Broca’s area early in brain development and... nothing happens? Human brain has specialized regions, but there's still a lot of flexibility in it. It isn't a hard fixed function system at all. A lung can't just start pumping blood to compensate for a heart issue, but similar things happen to brain regions. The regions can end up repurposed, and an impressive amount of damage can be routed around. A lot of the "brain damage" studies seem to point at a process not too dissimilar to ablation in artificial neural networks. You can null out some of the weights in a pretrained neural network, and that can fuck it up. But if you start fine-tuning the network afterwards, or train from scratch, with those weights still pinned to zero? The resulting performance can end up quite similar to a control case. A major difference is that human brain doesn't separate training from inference. Both are always happening - but the proportion varies. It may be nigh-impossible to fully "undo" some types of damage if it happens after a certain associated development window has closed, but easy enough if the damage happens beforehand.
- littlestymaar 1y ago
  > Damage Broca’s area early in brain development and... nothing happens? Citation needed. Cerebral plasticity is a thing, but its not magic either.
  - ACCount37 1y ago
    It's not magic, but it's just magic enough to conclusively disprove "brain regions are fixed function" - if information-theoretic reasons somehow weren't enough for you. Way too much weird re-routing and re-purposing can happen in the brain for that to be the case. Human brain implements a learning algorithm of some kind - neuroscientists disagree on the specifics, but not on the core notion. It doesn't ship with all the knowledge it needs, or anywhere close. To work well, it has to learn, and it has to learn by getting information from the environment.
    - littlestymaar 1y ago
      > It's not magic, but it's just magic enough to conclusively disprove "brain regions are fixed function" You cannot confidently disprove anything unless you can back your statement. > information-theoretic reasons somehow weren't enough for you. Your “Information-theoric reasoning” is completely pointless though. > Human brain implements a learning algorithm of some kind - neuroscientists disagree on the specifics, but not on the core notion. It doesn't ship with all the knowledge it needs, or anywhere close. To work well, it has to learn, and it has to learn by getting information from the environment Nobody said otherwise. But that doesn't mean everything is being learned either. There are many things a human is born with that it doesn't have to learn. (It's pretty obvious when you have kids: as primates humans are naturally attracted to climbing trees, and they will naturally collect stones and sticks, which is what primitive tools are made of).
      - ACCount37 1y ago
        And all of that "innate knowledge" still fits into under 1 gigabyte of compressed DNA. 1 gigabyte. That's the absolute limit of how much "innate knowledge" a human brain can have in it! Every single instinct, every learning algorithm, every innate behavior and every little cue a brain uses to build itself has to fit into a set of data just 1 gigabyte in size. Clearly, nature must have found some impressively large levers - to be able to build and initialize brain with 90 billion connected neurons in it off something this small.

ACCount37 1y ago

Damage Broca’s area early in brain development and... nothing happens? Human brain has specialized regions, but there's still a lot of flexibility in it. It isn't a hard fixed function system at all. A lung can't just start pumping blood to compensate for a heart issue, but similar things happen to brain regions. The regions can end up repurposed, and an impressive amount of damage can be routed around. A lot of the "brain damage" studies seem to point at a process not too dissimilar to ablation in artificial neural networks. You can null out some of the weights in a pretrained neural network, and that can fuck it up. But if you start fine-tuning the network afterwards, or train from scratch, with those weights still pinned to zero? The resulting performance can end up quite similar to a control case. A major difference is that human brain doesn't separate training from inference. Both are always happening - but the proportion varies. It may be nigh-impossible to fully "undo" some types of damage if it happens after a certain associated development window has closed, but easy enough if the damage happens beforehand.

littlestymaar 1y ago
> Damage Broca’s area early in brain development and... nothing happens? Citation needed. Cerebral plasticity is a thing, but its not magic either.
- ACCount37 1y ago
  It's not magic, but it's just magic enough to conclusively disprove "brain regions are fixed function" - if information-theoretic reasons somehow weren't enough for you. Way too much weird re-routing and re-purposing can happen in the brain for that to be the case. Human brain implements a learning algorithm of some kind - neuroscientists disagree on the specifics, but not on the core notion. It doesn't ship with all the knowledge it needs, or anywhere close. To work well, it has to learn, and it has to learn by getting information from the environment.
  - littlestymaar 1y ago
    > It's not magic, but it's just magic enough to conclusively disprove "brain regions are fixed function" You cannot confidently disprove anything unless you can back your statement. > information-theoretic reasons somehow weren't enough for you. Your “Information-theoric reasoning” is completely pointless though. > Human brain implements a learning algorithm of some kind - neuroscientists disagree on the specifics, but not on the core notion. It doesn't ship with all the knowledge it needs, or anywhere close. To work well, it has to learn, and it has to learn by getting information from the environment Nobody said otherwise. But that doesn't mean everything is being learned either. There are many things a human is born with that it doesn't have to learn. (It's pretty obvious when you have kids: as primates humans are naturally attracted to climbing trees, and they will naturally collect stones and sticks, which is what primitive tools are made of).
    - ACCount37 1y ago
      And all of that "innate knowledge" still fits into under 1 gigabyte of compressed DNA. 1 gigabyte. That's the absolute limit of how much "innate knowledge" a human brain can have in it! Every single instinct, every learning algorithm, every innate behavior and every little cue a brain uses to build itself has to fit into a set of data just 1 gigabyte in size. Clearly, nature must have found some impressively large levers - to be able to build and initialize brain with 90 billion connected neurons in it off something this small.