5 ms·

The estimate for how many operations per second the brain does is quite a wild guess. It starts out very reasonable, estimating the amount of information the ey

The estimate for how many operations per second the brain does is quite a wild guess. It starts out very reasonable, estimating the amount of information the eyes feed to the brain. But from there on it's really just a wild guess. We don't know how the brain processes visual input, we don't know what the fundamental unit of processing in the brain is, or if there is one.

nyrikki 3y ago
It is more than a wild guess, it is a guess based on the outdated perceptron model of the brain. Active dendrites the that can do operations like xor before anything reaches the soma, or use spike timing, once again before the soma are a couple of examples. SNNs, or spikey artificial NNs have been hitting limits of the computable numbers. Riddled basins as an example, which are sets with no open subsets. Basically any circle you can draw will always contain at least one point on a boundary set. https://arxiv.org/abs/1711.02160 https://arxiv.org/abs/1711.02160 This is a much stronger constraint than even classic chaos. While I am sure we will continue to try to make comparisons, wetwear and hardware are cheese and chalk. Both have things they are better at and neither is a direct replacement for the other.
hax0ron3 3y ago
It also blows my mind that that, even though DNA seems to just code for proteins and does not store a schematic for a brain in any way that we've been able to decipher so far, human eggs pretty reliably end up growing into people who 9 months after conception already have a bunch of stuff, including visual processing, working. Of course the DNA is not the only input, there is also the mother's body, the whole process of pregnancy, but I don't know how that contributes to the new baby being able to enter the world already intelligent. It's possible that I'm not aware of some breakthroughs on this topic, though.
- HarHarVeryFunny 3y ago
  DNA does a lot more than coding for proteins. It also controls how genes are expressed (how same gene does different things in different contexts) and how cells differentiate into different types (neurons, skin, muscle, etc) depending on location. In this way, DNA does control the blueprint for the brain, down to the initial generic wiring scheme of the cortex - six layers of differentiated neuron types, with a specific pattern of inter-connectivity. As a rough analogy, you could think of this DNA-controlled initial wiring scheme as something like an untrained transformer... the architecture is there, but won't do anything useful until it has been exposed to a lot of data, which will complete the wiring scheme (synapses = brain's model weights). The brain is going to start to learn as soon as it has sufficiently developed, which is certainly before birth, and newborn babies have been shown to already respond to things like their mothers voice which they were exposed to in the womb. Intelligence - ability to learn - is different from knowledge. A newborn baby has very little knowledge, but it does have intelligence since that comes from the brain architecture, hence ability to learn, that our DNA "encodes".
- viewtransform 3y ago
  The story of what we know so far about the development process is fascinating and captured in this book "Endless Forms Most Beautiful" by Sean B. Carroll (2005)
  - tambourine_man 3y ago
    Not to be confused with the Sean M Carroll
- idiotsecant 3y ago
  It is awesome, in the full nearly dreadfully amazing sense of the word.
- RajT88 3y ago
  I had a DevOps engineer describe their deployment automation system as a "Giant Rube Goldberg Machine". Which of course, all programming is a giant Rube Goldberg machine, if you think about it. If you take the "Goldberg" analogy to mean "Haphazard" and not well-thought out intentional structuring, I suppose it'd be more appropriate to compare a well-written application to a mechanical swiss watch. Living beings are more "Rube Goldberg" than "Swiss Watch" - by virtue that they came to being guided not by intention, but circumstance.
  - Nevermark 3y ago
    I worked on a 25 year project that stretched the tech many times in its life. On many occasions to get ambitious things to work despite a lack of good support, special subsystems were created with whatever wacky solution could be made to work, wrapped in sane surface API’s so the system as a whole could remain organized and accomplish the task well. Five or so years after any of these were created, they could have easily been replaced with more reasonable code. But they worked reliably and efficiently. Their API’s were sensible and stable. So they accumulated. If you reviewed all the code as a whole for the first time it was like finding a secret circus.
- nullc 3y ago
  newborns barely know how to see, they have to learn it. https://lozierinstitute.org/dive-deeper/the-newborn-senses-sight-and-eye-color/ https://lozierinstitute.org/dive-deeper/the-newborn-senses-s...
  - 8bitsrule 3y ago
    Along with the native language they're surrounded with ... and have to progressively decode all the intricacies thereof.
    - butlike 3y ago
      But the language processor is in place (and arguably robust enough to map any arbitrary language in the world)
- holoduke 3y ago
  Yes electricity. Just like when you open up a ssd and inspect it under a microscope, its very hard to physically identify software stored on that disc. Same with human cells. Probably some electric layer containing programs of the human body.
- kazinator 3y ago
  It's because nature never actually had to solve that problem: "how do we encode the structure we have here into genes and chromosomes so that it can procreate". The genes randomly mutate, and their result is subject to fitness testing. The fascinating thing about evolution is that it's not actually necessary. The "unevolved" creatures persist just fine. The continued survival of cockroaches and whatnot proves that evolving to human levels is superfluous.
- butlike 3y ago
  I'm thinking out loud here, but what if it's akin to a copy mechanism, or compression expander algorithm like unzipping a file? The Sperm carries the requisite DNA to activate the egg, which then just executes the body's Replicate() method. The sperm being required is so people don't spontaneously get pregnant. The DNA coding for proteins is just to build the replicate() method, which then starts copying attributes from the host's (read: mother's) body. Much like how there are differences in compression/expansion algorithms (zip, tar, gz, etc..), the sperm that initiates the replication/unpacking adds non-neutral variance, hence the genetic traits of both parents present in the child. Put a different way, why reinvent the wheel if I could just grab the memoized visual processing from the parent? Then, if a trait isn't present on the host, say, mid-digital hair, I (as the replication algorithm), "wouldn't know what I wouldn't know" which could perhaps be seeing as synonymous with "evolution."
  - sdwr 3y ago
    How does IVF work then? Or animals that lay eggs?
marcosdumay 3y ago
I remember people discovering that the retina neurons were actually in a very non-common and much less connected geometry than most of our neurons, what meant that the numbers on that page were (an unknown number of) orders of magnitude smaller than reality. But I have no idea what is known nowadays.
tim333 3y ago
Not really. The brain doesn't do operations per second like a computer. What he is guessing is that if so many operations per second can produce similar results to a certain volume of brain, in this case the retina, then maybe that holds for the rest of the brain. It seems to me a reasonable guess that appears to be proving approximately true.
cornholio 3y ago
I'm going to go against the grain here and hypothesize that the computational requirements for emulating human reason are much lower than commonly claimed, and probably within the reach of a high end GPU released in the last/next 10 years. If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans could traverse in centuries. Yet, their results are barely intelligent. The human neural network, on the other hand, stores and is able to recall an estimated average of a few bits per waking second for a decade or two, barely a few GB of useful storage [1]. A great deal of effort and substance of the brain is dedicated to compressing the sensory input into an internal representation useful for rational though - a problem much simplified for a text-fed neural network. The ability of blind and deaf people (or even blind-deaf) for rational reasoning show these ancillary tasks are not intrinsic to human reason. So things seem to indicate that what we lack is not hardware, but the right architectures and algorithms to unlock human level reason, the exceptionally tight self training loop of humans that can make sense of the world and even push humanity's understanding further using only a few GB of compressed training data. [1] LANDAUER TK. How much do people remember ‐ some estimates of the quantity of learned information in long‐term‐memory.
- sdwr 3y ago
  One GPU running continuously is arguable, the 10s of thousands that go into training LLM have definitely way over shot human processing flops. Evolution definitely still has us beat on optimization-work put into thinking. Some implemented hacks: - emotions as shortcuts for reasoning - learning through imitation - self-regulation through relation
- TheRoque 3y ago
  > If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans couldn't traverse in centuries. Yet, their results are barely intelligent Actually this is untrue. Yann Lecun himself said it. A kid at the age of 6 has had much much more data go through his brain (based on the visual data) than what any transformer has ingested. Moreover, words are already compressed data with a lot of lost information, nowhere near the raw experience about the world that our brains have. On top of that there's the evolutionary stuff.
naveen99 3y ago
Atleast it’s better / less futile than extrapolating universal expansion back in time to the Big Bang. but maybe not too much. Computers are already great peripherals for brains. Maybe brains will become peripherals for computers or become completely replaced.

nyrikki 3y ago

It is more than a wild guess, it is a guess based on the outdated perceptron model of the brain. Active dendrites the that can do operations like xor before anything reaches the soma, or use spike timing, once again before the soma are a couple of examples. SNNs, or spikey artificial NNs have been hitting limits of the computable numbers. Riddled basins as an example, which are sets with no open subsets. Basically any circle you can draw will always contain at least one point on a boundary set. https://arxiv.org/abs/1711.02160 https://arxiv.org/abs/1711.02160 This is a much stronger constraint than even classic chaos. While I am sure we will continue to try to make comparisons, wetwear and hardware are cheese and chalk. Both have things they are better at and neither is a direct replacement for the other.

hax0ron3 3y ago

It also blows my mind that that, even though DNA seems to just code for proteins and does not store a schematic for a brain in any way that we've been able to decipher so far, human eggs pretty reliably end up growing into people who 9 months after conception already have a bunch of stuff, including visual processing, working. Of course the DNA is not the only input, there is also the mother's body, the whole process of pregnancy, but I don't know how that contributes to the new baby being able to enter the world already intelligent. It's possible that I'm not aware of some breakthroughs on this topic, though.

HarHarVeryFunny 3y ago
DNA does a lot more than coding for proteins. It also controls how genes are expressed (how same gene does different things in different contexts) and how cells differentiate into different types (neurons, skin, muscle, etc) depending on location. In this way, DNA does control the blueprint for the brain, down to the initial generic wiring scheme of the cortex - six layers of differentiated neuron types, with a specific pattern of inter-connectivity. As a rough analogy, you could think of this DNA-controlled initial wiring scheme as something like an untrained transformer... the architecture is there, but won't do anything useful until it has been exposed to a lot of data, which will complete the wiring scheme (synapses = brain's model weights). The brain is going to start to learn as soon as it has sufficiently developed, which is certainly before birth, and newborn babies have been shown to already respond to things like their mothers voice which they were exposed to in the womb. Intelligence - ability to learn - is different from knowledge. A newborn baby has very little knowledge, but it does have intelligence since that comes from the brain architecture, hence ability to learn, that our DNA "encodes".
viewtransform 3y ago
The story of what we know so far about the development process is fascinating and captured in this book "Endless Forms Most Beautiful" by Sean B. Carroll (2005)
- tambourine_man 3y ago
  Not to be confused with the Sean M Carroll
idiotsecant 3y ago
It is awesome, in the full nearly dreadfully amazing sense of the word.
RajT88 3y ago
I had a DevOps engineer describe their deployment automation system as a "Giant Rube Goldberg Machine". Which of course, all programming is a giant Rube Goldberg machine, if you think about it. If you take the "Goldberg" analogy to mean "Haphazard" and not well-thought out intentional structuring, I suppose it'd be more appropriate to compare a well-written application to a mechanical swiss watch. Living beings are more "Rube Goldberg" than "Swiss Watch" - by virtue that they came to being guided not by intention, but circumstance.
- Nevermark 3y ago
  I worked on a 25 year project that stretched the tech many times in its life. On many occasions to get ambitious things to work despite a lack of good support, special subsystems were created with whatever wacky solution could be made to work, wrapped in sane surface API’s so the system as a whole could remain organized and accomplish the task well. Five or so years after any of these were created, they could have easily been replaced with more reasonable code. But they worked reliably and efficiently. Their API’s were sensible and stable. So they accumulated. If you reviewed all the code as a whole for the first time it was like finding a secret circus.
nullc 3y ago
newborns barely know how to see, they have to learn it. https://lozierinstitute.org/dive-deeper/the-newborn-senses-sight-and-eye-color/ https://lozierinstitute.org/dive-deeper/the-newborn-senses-s...
- 8bitsrule 3y ago
  Along with the native language they're surrounded with ... and have to progressively decode all the intricacies thereof.
  - butlike 3y ago
    But the language processor is in place (and arguably robust enough to map any arbitrary language in the world)
holoduke 3y ago
Yes electricity. Just like when you open up a ssd and inspect it under a microscope, its very hard to physically identify software stored on that disc. Same with human cells. Probably some electric layer containing programs of the human body.
kazinator 3y ago
It's because nature never actually had to solve that problem: "how do we encode the structure we have here into genes and chromosomes so that it can procreate". The genes randomly mutate, and their result is subject to fitness testing. The fascinating thing about evolution is that it's not actually necessary. The "unevolved" creatures persist just fine. The continued survival of cockroaches and whatnot proves that evolving to human levels is superfluous.
butlike 3y ago
I'm thinking out loud here, but what if it's akin to a copy mechanism, or compression expander algorithm like unzipping a file? The Sperm carries the requisite DNA to activate the egg, which then just executes the body's Replicate() method. The sperm being required is so people don't spontaneously get pregnant. The DNA coding for proteins is just to build the replicate() method, which then starts copying attributes from the host's (read: mother's) body. Much like how there are differences in compression/expansion algorithms (zip, tar, gz, etc..), the sperm that initiates the replication/unpacking adds non-neutral variance, hence the genetic traits of both parents present in the child. Put a different way, why reinvent the wheel if I could just grab the memoized visual processing from the parent? Then, if a trait isn't present on the host, say, mid-digital hair, I (as the replication algorithm), "wouldn't know what I wouldn't know" which could perhaps be seeing as synonymous with "evolution."
- sdwr 3y ago
  How does IVF work then? Or animals that lay eggs?

marcosdumay 3y ago

I remember people discovering that the retina neurons were actually in a very non-common and much less connected geometry than most of our neurons, what meant that the numbers on that page were (an unknown number of) orders of magnitude smaller than reality. But I have no idea what is known nowadays.

tim333 3y ago

Not really. The brain doesn't do operations per second like a computer. What he is guessing is that if so many operations per second can produce similar results to a certain volume of brain, in this case the retina, then maybe that holds for the rest of the brain. It seems to me a reasonable guess that appears to be proving approximately true.

cornholio 3y ago

I'm going to go against the grain here and hypothesize that the computational requirements for emulating human reason are much lower than commonly claimed, and probably within the reach of a high end GPU released in the last/next 10 years. If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans could traverse in centuries. Yet, their results are barely intelligent. The human neural network, on the other hand, stores and is able to recall an estimated average of a few bits per waking second for a decade or two, barely a few GB of useful storage [1]. A great deal of effort and substance of the brain is dedicated to compressing the sensory input into an internal representation useful for rational though - a problem much simplified for a text-fed neural network. The ability of blind and deaf people (or even blind-deaf) for rational reasoning show these ancillary tasks are not intrinsic to human reason. So things seem to indicate that what we lack is not hardware, but the right architectures and algorithms to unlock human level reason, the exceptionally tight self training loop of humans that can make sense of the world and even push humanity's understanding further using only a few GB of compressed training data. [1] LANDAUER TK. How much do people remember ‐ some estimates of the quantity of learned information in long‐term‐memory.

sdwr 3y ago
One GPU running continuously is arguable, the 10s of thousands that go into training LLM have definitely way over shot human processing flops. Evolution definitely still has us beat on optimization-work put into thinking. Some implemented hacks: - emotions as shortcuts for reasoning - learning through imitation - self-regulation through relation
TheRoque 3y ago
> If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans couldn't traverse in centuries. Yet, their results are barely intelligent Actually this is untrue. Yann Lecun himself said it. A kid at the age of 6 has had much much more data go through his brain (based on the visual data) than what any transformer has ingested. Moreover, words are already compressed data with a lot of lost information, nowhere near the raw experience about the world that our brains have. On top of that there's the evolutionary stuff.

naveen99 3y ago

Atleast it’s better / less futile than extrapolating universal expansion back in time to the Big Bang. but maybe not too much. Computers are already great peripherals for brains. Maybe brains will become peripherals for computers or become completely replaced.