8 ms·

"But human intelligence is limited by the size of the skull that houses the brain." When you think about it this way, it seems impossible that we haven't dupli

"But human intelligence is limited by the size of the skull that houses the brain." When you think about it this way, it seems impossible that we haven't duplicated the capability of the human brain in an airplane hangar somewhere. What's going on inside our heads that we can't mimic? That magical algorithm...

clanrebornx 8y ago
We know nothing about how it works, it seems to derive its information and results from somewhere else as if it's hooked some bigger brain ( which can't see ), so analyzing brain alone we don't find anything.
- zipzap324 8y ago
  This is inaccurate. We understand much of how it works, how vision, speech, etc work but we don't understand consciousness which is quite different.
  - arendtio 8y ago
    Well, maybe it just doesn't exist and my Roomba is as conscious as me.
    - Brifer 8y ago
      This made me ponder.
  - mattkrause 8y ago
    Nonsense. I’m an actual, working neuroscientist and if we’ve solved any of these things, it would be news to me (and everyone else at my institute). We have good, if coarse, knowledge of which structures are critical for which functions—-at least under some conditions. Our knowledge of how they do this is even cruder: neither the representations nor the algorithms are known with much certainty, let alone how they arise. Let me give you a very concrete example of where we are. There’s a small nemode called C. elegans. It’s about a millimeter long and has 302 neurons. We know its complete wiring diagram, its genome, and the origin and fate of every cell (not just the neurons) in its tiny, simple body. Its behavior has been studied extensively. And yet...we can’t accurately simulate the damn thing—-and it’s not like it does a lot to begin with. The human brain has about 86B neurons, and we know an awful lot less about them. Neither vision nor speech is remotely close to “solved” or understood. Consciousness, even in the very limited sense of “why do we fall asleep—-or need to?” is a mote off in the distance.
    - zipzap324 8y ago
      Where did I say anything about simulating? Are you really saying that we understand nothing about how vision works? Also a 10 second google search pulls up papers with plenty of details about vision in the brain so I don't know why you're talking about it as if its some great mystery. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4574956/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4574956/ A 1800 Page textbook on Vision Neuroscience. I'll leave it to HN to decide if we "Understand" Vision. Arbitrarily high requirements for understanding are being thrown about that would leave modern science in shambles if applied. https://mitpress.mit.edu/books/visual-neurosciences-2-vol-set https://mitpress.mit.edu/books/visual-neurosciences-2-vol-se...
      - thereisnospork 8y ago
        We understand lots about how the brain behaves, we understand next to nil about how it behaves that way. we can't describe how vision, eg, is processed at a molecular or even cellular level, ergo we know nothing.
      - mattkrause 8y ago
        I have a PhD in visual neuroscience, so yeah, I’m pretty comfortable saying we don’t know how it works. We know a lot of facts, and we have some ideas about how various small things are implemented, but in terms of grand unifying theories, we’re nowhere close. For example, suppose I showed you two gratings (think zebra stripes): a small patch and a larger one. Under some circumstances, you’ll have a harder time determining which way the big patch is oriented vs. the small one. This is true even though there’s extra information in the big patch. We think this is related to a phenomena called surround suppression, but they’re not exactly the same....and no one can agree on how surround suppression is implemented, let alone what it’s good for. This happens in primary visual cortex, which is probably the simplest—-and most extensively studied—of the visual cortical areas.
        0-_-0 8y ago
        It's quite unrelated, but I've been wondering something about neuroscience grammar: Why is it "in primary visual cortex" and not "in _the_ primary visual cortex"?
  - wuliwong 8y ago
    You responded to hyperbole with with a vague and imprecise statement. I will admit it is more accurate since "much" is somewhere between "everything" and "nothing".
zipzap324 8y ago
Modern supercomputers are still a factor of 10 off of the brains compute power(~1 Exaflop) but I don't think anyone in the field believes that you suddenly get consciousness once you reach a critical mass of compute. It's clearly a software problem.
- melling 8y ago
  A factor of 10? That's it. It's as good as done within 5-7 years. https://www.sciencemag.org/news/2018/02/racing-match-chinas-growing-computer-power-us-outlines-design-exascale-computer https://www.sciencemag.org/news/2018/02/racing-match-chinas-...
- ZhuanXia 8y ago
  Estimates vary over many orders of magnitude: https://aiimpacts.org/brain-performance-in-flops/ https://aiimpacts.org/brain-performance-in-flops/
  - zipzap324 8y ago
    ~1 Exaflop leaves room for error.
    - ZhuanXia 8y ago
      I suppose my point is I distrust your certainty. It's quite possible 20 petaflops may be enough. Maybe we will need many exaflops. We don't really know.
    - tim333 8y ago
      Moravec's estimate which seems quite well reasoned puts it much lower, about 100 terraflops which you can now get in a GPU https://www.scan.co.uk/products/pny-nvidia-tesla-v100-16-gb-cowos-hbm2-with-ecc-pci-e-30-x16 https://www.scan.co.uk/products/pny-nvidia-tesla-v100-16-gb-... It's true they have not suddenly become conscious.
modzu 8y ago
this is wrong. look at the size of an elephant brain. they are smart, but not smarter than humans. similarly, bird brains are tiny, but birds have shown significant intelligence, crows in particular have been well studied.
- melling 8y ago
  The point is that we're trying to duplicate something with the ability of the human brain, and we aren't constrained by size. We can cheat (e.g. 1,000,000 times bigger with 1,000,000 times the power, etc). We're just missing an algorithm or two.
  - modzu 8y ago
    ah yes, it's true we can keep adding on processors and memory until we go beyond what's in the brain (i think currently deep mind has ~ 11 billion neurons to our 100 billion), and indeed so far it's helping, but my point is that it's not at all clear that "more = better". there is more complexity involved than simply scale. the machine could become schizophrenic for exmaple (seriously).
    - deleted 8y ago
      [deleted]
    - riku_iki 8y ago
      > in the brain you focus only on single brain, when really we need to look at many trillions brains and experiments with complex reward function(real world): billion years of evolution * billions of various creatures born and died within every year. All this giant sequence of experiments converged to current human brain. In ML terms: to achieve result similar to human brain we need to run that many hyperparameter optimization trials. Or find some better shortcuts than human brain biological structure.
    - vl 8y ago
      This is not a best way to compare it: brain neurons do orders of magnitudes less operations per second. Current Google's TPU clusters are 10 petaflops, with new ones being 1 exoflops. We can estimate 100Hz frequency for brain, so 100 * 10^11 = 10^13 "neuron operations". Let's estimate 1000 flops to simulate single neuron. So it gives us 10^16 flops for brain (aka 10 petaplops). I.e. Google's new deployments are more powerful than single brain.
mac01021 8y ago
Similar things have been or are being tried (eg https://en.wikipedia.org/wiki/Blue_Brain_Project https://en.wikipedia.org/wiki/Blue_Brain_Project). But the _precise_ capabilities of a human brain are not actually what we want. For starters, an infant's brain does not have any immediately valuable capabilities. After that brain is exposed for several years to signals propagating through the brain's host body from the surrounding environment, it has developed many interesting capabilities. But those capabilities are only meaningful in the context of the input stream that the brain has learned to interpret. So if you want your software simulation of a brain, running on a hangar-sized computing cluster, to perform human grade cognition, then you'll have to provide it with a signal as rich as and of the same form as the signal that we receive on a continual basis through our 1 billion sensory cells (optical, auditory, proprioceptive, etc). And in order to supply that signal in a realistic way, you'll have to simulate the environment in such a way that it responds to motor output from the simulated brain. (Or you can use the real environment, but then you have to have the brain operate a complete synthetic human body). All this is a tremendous technical challenge, outlandishly expensive and, even when achieved, does not immediately enhance our understanding of how naturally intelligent systems process information. Nor does it provide us with a means to construct specialized intelligent agents that operate in the world, whether autonomous vehicles, burger flippers, surgeons, or stock brokers.
- arisAlexis 8y ago
  how about watching all movies and all internet media?
mattkrause 8y ago
If you poke at a real brain, it’s almost fractally complex. Single ion channels can have surprisingly complicated behaviors that depend on their current state and past history. Individual neurons contain tons of these channels, and can do a lot of powerful computation on their own. Of course, there are 86 billion neurons and combinatorically more connections between them. That’s just the neurons too; God only knows what the glia cells, which outnumber them 10:1, are doing but they’re a lot less passive than many have thought. On top of this, there’s a whole separate but overlaid network of neuromodulators (hormones, nitric oxide, etc). Electric fields produced by some neurons may even influence the activity of others. None of this is static, either. Things change on timescales ranging from milliseconds to years, and in response to all sorts of external stimuli. The brain is bonkers.
- est31 8y ago
  The coolest part about this is probably how little energy it needs. It's about 20 Watts. Current node sizes are already smaller than axon diameters. If it weren't for power, we could probably scale up manufacturing processes of current semiconductor technology to build giant room sized chips but we simply wouldn't be able to cool those things. Instead we are forced to wrap a ton of metal around them and plastic and air and provide extensive cooling.
  - FartyMcFarter 8y ago
    Is it true that a big part of this efficiency is due to using analog signals and not digital? I recall reading that, but it's far from my area of expertise. Digital has many advantages: a digital Einstein could be replicated perfectly, not so for an analog Einstein.
- wuliwong 8y ago
  Well said. This is my line of thinking when comparing current AI to the human brain.
narrator 8y ago
Protein folding is still an exponential time algorithm when done inside a computer and biological systems do this in constant time, massively, in parallel. Determining whether a molecule is an agonist takes a long time to calculate. I've heard the complexity is O(N^3). Biological systems do this in constant time, trillions of times a second in parallel. If you could simulate biological systems easily, you could do drug development completely inside a computer.
- alphagrep12345 8y ago
  So are you saying that we need not build bigger networks with huge data, but rather have better algorithms with significant parallelization? Sounds like we're back to symbolic AI again!
  - narrator 8y ago
    New computing architectures like quantum computing could simulate these systems far more efficiently than classical architectures -- in theory. The engineering still needs more work though. Also, new AI techniques are good at finding good enough equivalents to exactly replicating human cognition for many things, but that might get harder as the problems become even more general.
- fwip 8y ago
  I'm concerned that you're ascribing unique characteristics to biological systems, when in fact we see the same characteristics in almost any natural system. Biological systems don't solve protein folding. Proteins fold in biological systems. Fluids take an enormous amount of computing power to simulate correctly. But water isn't smart, it just does what water does. Heck, the N-body problem is a classic O(n^2) algorithm, but we don't say that the planets and stars are "solving" it.
  - narrator 8y ago
    These techniques have already been used to solve problems using DNA computing [1]. Digital circuits don't solve anything either, they are just electrons flowing around mediated by semiconductors after all and the results of their computation are just measurements of electrical charge values after those physical systems have had time to flow around. As long as you can set the input and get a consistent output and measure that output you can compute with it. To use your logic: Large integrated circuits take an enormous amount of computing power to simulate correctly. But integrated circuits aren't smart, they just do what integrated circuits do. [1]https://en.m.wikipedia.org/wiki/DNA_computing https://en.m.wikipedia.org/wiki/DNA_computing
Symmetry 8y ago
Imagine if marsupials, not limited by the size of the birth canals, had produced an intelligent species. Or birds. Especially since birds have the same linear relationships between brain size and neuron count that primates do.
gumby 8y ago
Think of it as a 100 Hz computer with 80 giganodes and a fanout from those nodes of up to 10^5. That's a lot of computation!
gumby 8y ago
> "But human intelligence is limited by the size of the skull that houses the brain." It's actually not. Humans are unusual in that they can teach each other and institutional knowledge can span generations. In addition, just as your speed of travel is not limited by the length of your legs and the ATP cycle, your intelligence is not limited by your brain.
Barrin92 8y ago
It's not necessarily about a magical algorithm but about the richness of information that can be processed by analog, biological devices. The amount of information stored and being processed by single individual cells is almost unimaginable. Individual brain cells are likely able to learn high level abstract features store memory, modulate responses in intensity and length through thousands of transmitters and so on. Single celled organisms, like amoeba, possess the ability to emulating 'hunting' and other complex behaviour.

clanrebornx 8y ago

We know nothing about how it works, it seems to derive its information and results from somewhere else as if it's hooked some bigger brain ( which can't see ), so analyzing brain alone we don't find anything.

zipzap324 8y ago
This is inaccurate. We understand much of how it works, how vision, speech, etc work but we don't understand consciousness which is quite different.
- arendtio 8y ago
  Well, maybe it just doesn't exist and my Roomba is as conscious as me.
  - Brifer 8y ago
    This made me ponder.
- mattkrause 8y ago
  Nonsense. I’m an actual, working neuroscientist and if we’ve solved any of these things, it would be news to me (and everyone else at my institute). We have good, if coarse, knowledge of which structures are critical for which functions—-at least under some conditions. Our knowledge of how they do this is even cruder: neither the representations nor the algorithms are known with much certainty, let alone how they arise. Let me give you a very concrete example of where we are. There’s a small nemode called C. elegans. It’s about a millimeter long and has 302 neurons. We know its complete wiring diagram, its genome, and the origin and fate of every cell (not just the neurons) in its tiny, simple body. Its behavior has been studied extensively. And yet...we can’t accurately simulate the damn thing—-and it’s not like it does a lot to begin with. The human brain has about 86B neurons, and we know an awful lot less about them. Neither vision nor speech is remotely close to “solved” or understood. Consciousness, even in the very limited sense of “why do we fall asleep—-or need to?” is a mote off in the distance.
  - zipzap324 8y ago
    Where did I say anything about simulating? Are you really saying that we understand nothing about how vision works? Also a 10 second google search pulls up papers with plenty of details about vision in the brain so I don't know why you're talking about it as if its some great mystery. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4574956/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4574956/ A 1800 Page textbook on Vision Neuroscience. I'll leave it to HN to decide if we "Understand" Vision. Arbitrarily high requirements for understanding are being thrown about that would leave modern science in shambles if applied. https://mitpress.mit.edu/books/visual-neurosciences-2-vol-set https://mitpress.mit.edu/books/visual-neurosciences-2-vol-se...
    - thereisnospork 8y ago
      We understand lots about how the brain behaves, we understand next to nil about how it behaves that way. we can't describe how vision, eg, is processed at a molecular or even cellular level, ergo we know nothing.
    - mattkrause 8y ago
      I have a PhD in visual neuroscience, so yeah, I’m pretty comfortable saying we don’t know how it works. We know a lot of facts, and we have some ideas about how various small things are implemented, but in terms of grand unifying theories, we’re nowhere close. For example, suppose I showed you two gratings (think zebra stripes): a small patch and a larger one. Under some circumstances, you’ll have a harder time determining which way the big patch is oriented vs. the small one. This is true even though there’s extra information in the big patch. We think this is related to a phenomena called surround suppression, but they’re not exactly the same....and no one can agree on how surround suppression is implemented, let alone what it’s good for. This happens in primary visual cortex, which is probably the simplest—-and most extensively studied—of the visual cortical areas.
      - 0-_-0 8y ago
        It's quite unrelated, but I've been wondering something about neuroscience grammar: Why is it "in primary visual cortex" and not "in _the_ primary visual cortex"?
- wuliwong 8y ago
  You responded to hyperbole with with a vague and imprecise statement. I will admit it is more accurate since "much" is somewhere between "everything" and "nothing".

zipzap324 8y ago

Modern supercomputers are still a factor of 10 off of the brains compute power(~1 Exaflop) but I don't think anyone in the field believes that you suddenly get consciousness once you reach a critical mass of compute. It's clearly a software problem.

melling 8y ago
A factor of 10? That's it. It's as good as done within 5-7 years. https://www.sciencemag.org/news/2018/02/racing-match-chinas-growing-computer-power-us-outlines-design-exascale-computer https://www.sciencemag.org/news/2018/02/racing-match-chinas-...
ZhuanXia 8y ago
Estimates vary over many orders of magnitude: https://aiimpacts.org/brain-performance-in-flops/ https://aiimpacts.org/brain-performance-in-flops/
- zipzap324 8y ago
  ~1 Exaflop leaves room for error.
  - ZhuanXia 8y ago
    I suppose my point is I distrust your certainty. It's quite possible 20 petaflops may be enough. Maybe we will need many exaflops. We don't really know.
  - tim333 8y ago
    Moravec's estimate which seems quite well reasoned puts it much lower, about 100 terraflops which you can now get in a GPU https://www.scan.co.uk/products/pny-nvidia-tesla-v100-16-gb-cowos-hbm2-with-ecc-pci-e-30-x16 https://www.scan.co.uk/products/pny-nvidia-tesla-v100-16-gb-... It's true they have not suddenly become conscious.

modzu 8y ago

this is wrong. look at the size of an elephant brain. they are smart, but not smarter than humans. similarly, bird brains are tiny, but birds have shown significant intelligence, crows in particular have been well studied.

melling 8y ago
The point is that we're trying to duplicate something with the ability of the human brain, and we aren't constrained by size. We can cheat (e.g. 1,000,000 times bigger with 1,000,000 times the power, etc). We're just missing an algorithm or two.
- modzu 8y ago
  ah yes, it's true we can keep adding on processors and memory until we go beyond what's in the brain (i think currently deep mind has ~ 11 billion neurons to our 100 billion), and indeed so far it's helping, but my point is that it's not at all clear that "more = better". there is more complexity involved than simply scale. the machine could become schizophrenic for exmaple (seriously).
  - deleted 8y ago
    [deleted]
  - riku_iki 8y ago
    > in the brain you focus only on single brain, when really we need to look at many trillions brains and experiments with complex reward function(real world): billion years of evolution * billions of various creatures born and died within every year. All this giant sequence of experiments converged to current human brain. In ML terms: to achieve result similar to human brain we need to run that many hyperparameter optimization trials. Or find some better shortcuts than human brain biological structure.
  - vl 8y ago
    This is not a best way to compare it: brain neurons do orders of magnitudes less operations per second. Current Google's TPU clusters are 10 petaflops, with new ones being 1 exoflops. We can estimate 100Hz frequency for brain, so 100 * 10^11 = 10^13 "neuron operations". Let's estimate 1000 flops to simulate single neuron. So it gives us 10^16 flops for brain (aka 10 petaplops). I.e. Google's new deployments are more powerful than single brain.

mac01021 8y ago

Similar things have been or are being tried (eg https://en.wikipedia.org/wiki/Blue_Brain_Project https://en.wikipedia.org/wiki/Blue_Brain_Project). But the _precise_ capabilities of a human brain are not actually what we want. For starters, an infant's brain does not have any immediately valuable capabilities. After that brain is exposed for several years to signals propagating through the brain's host body from the surrounding environment, it has developed many interesting capabilities. But those capabilities are only meaningful in the context of the input stream that the brain has learned to interpret. So if you want your software simulation of a brain, running on a hangar-sized computing cluster, to perform human grade cognition, then you'll have to provide it with a signal as rich as and of the same form as the signal that we receive on a continual basis through our 1 billion sensory cells (optical, auditory, proprioceptive, etc). And in order to supply that signal in a realistic way, you'll have to simulate the environment in such a way that it responds to motor output from the simulated brain. (Or you can use the real environment, but then you have to have the brain operate a complete synthetic human body). All this is a tremendous technical challenge, outlandishly expensive and, even when achieved, does not immediately enhance our understanding of how naturally intelligent systems process information. Nor does it provide us with a means to construct specialized intelligent agents that operate in the world, whether autonomous vehicles, burger flippers, surgeons, or stock brokers.

arisAlexis 8y ago
how about watching all movies and all internet media?

mattkrause 8y ago

If you poke at a real brain, it’s almost fractally complex. Single ion channels can have surprisingly complicated behaviors that depend on their current state and past history. Individual neurons contain tons of these channels, and can do a lot of powerful computation on their own. Of course, there are 86 billion neurons and combinatorically more connections between them. That’s just the neurons too; God only knows what the glia cells, which outnumber them 10:1, are doing but they’re a lot less passive than many have thought. On top of this, there’s a whole separate but overlaid network of neuromodulators (hormones, nitric oxide, etc). Electric fields produced by some neurons may even influence the activity of others. None of this is static, either. Things change on timescales ranging from milliseconds to years, and in response to all sorts of external stimuli. The brain is bonkers.

est31 8y ago
The coolest part about this is probably how little energy it needs. It's about 20 Watts. Current node sizes are already smaller than axon diameters. If it weren't for power, we could probably scale up manufacturing processes of current semiconductor technology to build giant room sized chips but we simply wouldn't be able to cool those things. Instead we are forced to wrap a ton of metal around them and plastic and air and provide extensive cooling.
- FartyMcFarter 8y ago
  Is it true that a big part of this efficiency is due to using analog signals and not digital? I recall reading that, but it's far from my area of expertise. Digital has many advantages: a digital Einstein could be replicated perfectly, not so for an analog Einstein.
wuliwong 8y ago
Well said. This is my line of thinking when comparing current AI to the human brain.

narrator 8y ago

Protein folding is still an exponential time algorithm when done inside a computer and biological systems do this in constant time, massively, in parallel. Determining whether a molecule is an agonist takes a long time to calculate. I've heard the complexity is O(N^3). Biological systems do this in constant time, trillions of times a second in parallel. If you could simulate biological systems easily, you could do drug development completely inside a computer.

alphagrep12345 8y ago
So are you saying that we need not build bigger networks with huge data, but rather have better algorithms with significant parallelization? Sounds like we're back to symbolic AI again!
- narrator 8y ago
  New computing architectures like quantum computing could simulate these systems far more efficiently than classical architectures -- in theory. The engineering still needs more work though. Also, new AI techniques are good at finding good enough equivalents to exactly replicating human cognition for many things, but that might get harder as the problems become even more general.
fwip 8y ago
I'm concerned that you're ascribing unique characteristics to biological systems, when in fact we see the same characteristics in almost any natural system. Biological systems don't solve protein folding. Proteins fold in biological systems. Fluids take an enormous amount of computing power to simulate correctly. But water isn't smart, it just does what water does. Heck, the N-body problem is a classic O(n^2) algorithm, but we don't say that the planets and stars are "solving" it.
- narrator 8y ago
  These techniques have already been used to solve problems using DNA computing [1]. Digital circuits don't solve anything either, they are just electrons flowing around mediated by semiconductors after all and the results of their computation are just measurements of electrical charge values after those physical systems have had time to flow around. As long as you can set the input and get a consistent output and measure that output you can compute with it. To use your logic: Large integrated circuits take an enormous amount of computing power to simulate correctly. But integrated circuits aren't smart, they just do what integrated circuits do. [1]https://en.m.wikipedia.org/wiki/DNA_computing https://en.m.wikipedia.org/wiki/DNA_computing

Symmetry 8y ago

Imagine if marsupials, not limited by the size of the birth canals, had produced an intelligent species. Or birds. Especially since birds have the same linear relationships between brain size and neuron count that primates do.

gumby 8y ago

Think of it as a 100 Hz computer with 80 giganodes and a fanout from those nodes of up to 10^5. That's a lot of computation!

gumby 8y ago

> "But human intelligence is limited by the size of the skull that houses the brain." It's actually not. Humans are unusual in that they can teach each other and institutional knowledge can span generations. In addition, just as your speed of travel is not limited by the length of your legs and the ATP cycle, your intelligence is not limited by your brain.

Barrin92 8y ago

It's not necessarily about a magical algorithm but about the richness of information that can be processed by analog, biological devices. The amount of information stored and being processed by single individual cells is almost unimaginable. Individual brain cells are likely able to learn high level abstract features store memory, modulate responses in intensity and length through thousands of transmitters and so on. Single celled organisms, like amoeba, possess the ability to emulating 'hunting' and other complex behaviour.