6 ms·

>"Intelligent machines don’t have to model all the complexity of biological neurons, but the capabilities enabled by dendrites and learning by rewiring are esse

>"Intelligent machines don’t have to model all the complexity of biological neurons, but the capabilities enabled by dendrites and learning by rewiring are essential. These capabilities will need to be in future AI systems." This is the fundamental challenge that we face. Our ability to build AI that can emulate human thought is not limited by a poor understanding of the brain. Our current theoretical understanding of how human cognition arises from neural processes is probably close to a level sufficient to build human level AI. What limits our progress is the staggering computational demand of simulating a massive network of highly dynamic units. Shortcuts, simplifications and clever algorithms will only get us so far. At this point, processing power rather than theoretical understanding is the limiting factor.

nabla9 9y ago
>Our current theoretical understanding of how human cognition arises from neural processes is probably close to a level sufficient to build human level AI. This is not true at all. The brain of C. elegans (roundworm) has been mapped exactly (connectome is known) and we know it's 302 neurons and 8000 synapses well (it has just 959 cells total) but we still can't fully understand how its primitive brain works. It don't even have spiking neurons and it's still a mystery. It would be relatively straight forward to simulate. There is even software for doing it. To fully understand how even simple brain works, we must understand gene expression inside brain cells, role of somatic brain mosaicism inside brain, brain chemistry, neural connectome, how cortical columns work, neural coding, etc. When you get all this right, you need to fine tune it. Hyperparameter optimization for human level AI so that it's not epileptic, autistic, schizophrenic, manic, idiot or AI equivalent of these and million other things must be really hard process. --- http://www.openworm.org/ http://www.openworm.org/ http://bluebrain.epfl.ch/page-52741-en.html http://bluebrain.epfl.ch/page-52741-en.html http://cajalbbp.cesvima.upm.es/ http://cajalbbp.cesvima.upm.es/
- grondilu 9y ago
  Thanks for reminding us of openworm. The fact that scientists are still working on simulating an animal with barely a thousand cells surely relativizes claims about simulating a mammalian brain, let alone a human one. Frankly, it's even kind of discouraging.
  - jacquesm 9y ago
    It's one of the reasons I totally do not believe in https://en.wikipedia.org/wiki/Human_Brain_Project https://en.wikipedia.org/wiki/Human_Brain_Project What I do believe is that once openworm achieves its goals that progress might become very rapid indeed.
  - Houshalter 9y ago
    Meh, evolution is known to write utterly incomprehensible spaghetti code. It's really not surprising that reverse engineering it is so hard. Write a very simple genetic algorithm to evolve artificial neural networks or computer code. Often it can solve simple tasks in creative ways. But trying to understand the output is usually a nightmare. It usually comes up with ridiculously convoluted and insane ways of doing things. E.g; https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/ >The plucky chip was utilizing only thirty-seven of its one hundred logic gates, and most of them were arranged in a curious collection of feedback loops. Five individual logic cells were functionally disconnected from the rest— with no pathways that would allow them to influence the output— yet when the researcher disabled any one of them the chip lost its ability to discriminate the tones. Furthermore, the final program did not work reliably when it was loaded onto other FPGAs of the same type. Now imagine a system many times larger and more sophisticated than that, that evolution has been hacking on for hundreds of millions of years. Doesn't mean we aren't close to human level AI though. We didn't completely reverse engineer birds before building airplanes, or horses before building cars.
- emcq 9y ago
  As a glimmer of hope, consider this: it may be easier to decode a much larger system with redundancies than the highly evolved and tightly optimized 302 neuron system. The C. elegans has a small number of heterogenous neurons. Some are quite task specific. The human brain, on the other hand, has ~100billion. While they do not all have the same structure, within specific regions you will see repeated patterns of homogeneity and redundant encodings. It may be a harder problem to start with the small system than the large one.
- Seanny123 9y ago
  I think it's more important to choose the right level of detail for what you're modelling. This is covered in "The Use and Abuse of Large Scale Brain Models" [1]. [1] https://github.com/arvoelke/delay2017/blob/master/delay2017.pdf https://github.com/arvoelke/delay2017/blob/master/delay2017....
eli_gottlieb 9y ago
>Our current theoretical understanding of how human cognition arises from neural processes is probably close to a level sufficient to build human level AI. Please state that understanding.
- meri_dian 9y ago
  Generally: We have a good idea of how neurons interact on a low level; dendritic connections among themselves. At a higher level, we know the brain regions and how they are connected to one another. Glial cells are still a bit of a mystery but work is advancing rapidly on that front.
  - jdmichal 9y ago
    And the whole slew of chemicals flying around that effect pretty much every interaction in the entire system.
  - EGreg 9y ago
    I see. Humans are composed of cells. Socities are composed of humans. Now we know everything about how they work! Umm ... :)
  - eli_gottlieb 9y ago
    I second what EGreg said. You have the additional problem that "brain regions" aren't composed of cells, they're composed of circuits, and neurons themselves come in multiple kinds that have different roles in neural circuits. Then you get the problem that saying, "it's circuits built into regions built out of neurons" doesn't make any predictions about the system as a whole. So, at the very least, this stratospherically high-level overview is incomplete.
computerex 9y ago
> Our current theoretical understanding of how human cognition arises from neural processes is probably close to a level sufficient to build human level AI. No. Not even remotely close. If that were true we'd be able to emulate the behavior somewhat of simpler organisms that don't have the connectivity of a human brain. The fact of the matter is, we simply don't understand cognition. I'd say the challenge in realizing strong AI right now is both a lack of understanding of the brain as well as emulating its mode of computation, which is different from a traditional digital computer.
jacquesm 9y ago
> At this point, processing power rather than theoretical understanding is the limiting factor. No. And you can clearly show this by reasoning like this: if what you say is true then we would be able to simulate a human cognition system but simply slower than real time. The fact that we can't even begin to do that no matter at what speed we would like to set this in motion proves that there is no such thing as sufficient theoretical understanding. To state that even stronger: we are nowhere near close to such understanding. We have roughly the kind of knowledge that you'd be able to glean by taking a hacksaw to a computer, figuring out that there is such a thing as a non-linear element and that you could use this for computation and maybe a basic digital circuit or two. Immensely useful, but not the level of understanding that we would like to have.
dhfhduk 9y ago
I do research in the behavioral sciences. I've published a little on analysis of neural activity, although it's not my area of specialty, but I do specialize in analyses that are cognate to DL models and are often covered in textbooks on DL along with classical DL models. My sense is that the level of understanding of how human cognition works mathematically speaking is roughly similar to our understanding of how DL works in computer science, in the sense that if you asked a cognitive neuroscientist or psychologist how someone classifies cats versus non-cats, you'd get an answer that would seem pretty similar to what you'd get from a computer scientist. The behavioral scientist might go into a lot more detail about certain issues, but that's because the biology is so entertwined. However, I'd also argue that we really don't know much about how human (or any animal) cognition works, and I'd also argue that our understanding of DL is fairly poor, in that a lot of it is tinkering and seeing what happens, without a deep understanding of why it works. There isn't a theory of DL in the same way that there's a Martin-Lof theory of randomness, or a Kolmogorov theory of algorithmic complexity, or a Fisherian model of inference. Also, the sort of tasks currently involved in AI research is a tiny subset of what you encounter in neuroscience and psychology. Most of what is a hot topic in comp sci would basically be classified as perceptual tasks in human behavioral science, maybe at a slightly higher level, and maybe motor control. That leaves things like conscious versus nonconscious processing, reasoning, the role of emotion in decision-making, uncertainty valuation, creativity, etc. etc. etc. I agree that the processing power is an issue but it's only part of the puzzle. One thing that illustrates the complexity of the issues involved, and how we've only begun to scratch the surface, is the article's assertion that comp sci should borrow more the idea of sparse representations from cognitive neuroscience. I thought that was interesting, because in a lot of ways, one of the major trends in the last ten years in human neuroscience is away from this "sparseness" idea. It was a common assumption maybe 15 years ago, but now people routinely get excoriated for invoking that idea. The current paradigm is one more where a lot of pathways/circuits are being recruited simultaneously. Statements like "you might use 10,000 neurons of which 100 are active" would lead to ridicule. The intuitive way of explaining the problem is that even while your brain is trying to decide if you're perceiving a cat versus something else, it's also processing the consequences of that decision along about 10 different dimensions, the implications for the rest of the stimuli coming in, along with a number of other things we just don't understand.

nabla9 9y ago

>Our current theoretical understanding of how human cognition arises from neural processes is probably close to a level sufficient to build human level AI. This is not true at all. The brain of C. elegans (roundworm) has been mapped exactly (connectome is known) and we know it's 302 neurons and 8000 synapses well (it has just 959 cells total) but we still can't fully understand how its primitive brain works. It don't even have spiking neurons and it's still a mystery. It would be relatively straight forward to simulate. There is even software for doing it. To fully understand how even simple brain works, we must understand gene expression inside brain cells, role of somatic brain mosaicism inside brain, brain chemistry, neural connectome, how cortical columns work, neural coding, etc. When you get all this right, you need to fine tune it. Hyperparameter optimization for human level AI so that it's not epileptic, autistic, schizophrenic, manic, idiot or AI equivalent of these and million other things must be really hard process. --- http://www.openworm.org/ http://www.openworm.org/ http://bluebrain.epfl.ch/page-52741-en.html http://bluebrain.epfl.ch/page-52741-en.html http://cajalbbp.cesvima.upm.es/ http://cajalbbp.cesvima.upm.es/

grondilu 9y ago
Thanks for reminding us of openworm. The fact that scientists are still working on simulating an animal with barely a thousand cells surely relativizes claims about simulating a mammalian brain, let alone a human one. Frankly, it's even kind of discouraging.
- jacquesm 9y ago
  It's one of the reasons I totally do not believe in https://en.wikipedia.org/wiki/Human_Brain_Project https://en.wikipedia.org/wiki/Human_Brain_Project What I do believe is that once openworm achieves its goals that progress might become very rapid indeed.
- Houshalter 9y ago
  Meh, evolution is known to write utterly incomprehensible spaghetti code. It's really not surprising that reverse engineering it is so hard. Write a very simple genetic algorithm to evolve artificial neural networks or computer code. Often it can solve simple tasks in creative ways. But trying to understand the output is usually a nightmare. It usually comes up with ridiculously convoluted and insane ways of doing things. E.g; https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/ >The plucky chip was utilizing only thirty-seven of its one hundred logic gates, and most of them were arranged in a curious collection of feedback loops. Five individual logic cells were functionally disconnected from the rest— with no pathways that would allow them to influence the output— yet when the researcher disabled any one of them the chip lost its ability to discriminate the tones. Furthermore, the final program did not work reliably when it was loaded onto other FPGAs of the same type. Now imagine a system many times larger and more sophisticated than that, that evolution has been hacking on for hundreds of millions of years. Doesn't mean we aren't close to human level AI though. We didn't completely reverse engineer birds before building airplanes, or horses before building cars.
emcq 9y ago
As a glimmer of hope, consider this: it may be easier to decode a much larger system with redundancies than the highly evolved and tightly optimized 302 neuron system. The C. elegans has a small number of heterogenous neurons. Some are quite task specific. The human brain, on the other hand, has ~100billion. While they do not all have the same structure, within specific regions you will see repeated patterns of homogeneity and redundant encodings. It may be a harder problem to start with the small system than the large one.
Seanny123 9y ago
I think it's more important to choose the right level of detail for what you're modelling. This is covered in "The Use and Abuse of Large Scale Brain Models" [1]. [1] https://github.com/arvoelke/delay2017/blob/master/delay2017.pdf https://github.com/arvoelke/delay2017/blob/master/delay2017....

eli_gottlieb 9y ago

>Our current theoretical understanding of how human cognition arises from neural processes is probably close to a level sufficient to build human level AI. Please state that understanding.

meri_dian 9y ago
Generally: We have a good idea of how neurons interact on a low level; dendritic connections among themselves. At a higher level, we know the brain regions and how they are connected to one another. Glial cells are still a bit of a mystery but work is advancing rapidly on that front.
- jdmichal 9y ago
  And the whole slew of chemicals flying around that effect pretty much every interaction in the entire system.
- EGreg 9y ago
  I see. Humans are composed of cells. Socities are composed of humans. Now we know everything about how they work! Umm ... :)
- eli_gottlieb 9y ago
  I second what EGreg said. You have the additional problem that "brain regions" aren't composed of cells, they're composed of circuits, and neurons themselves come in multiple kinds that have different roles in neural circuits. Then you get the problem that saying, "it's circuits built into regions built out of neurons" doesn't make any predictions about the system as a whole. So, at the very least, this stratospherically high-level overview is incomplete.

computerex 9y ago

> Our current theoretical understanding of how human cognition arises from neural processes is probably close to a level sufficient to build human level AI. No. Not even remotely close. If that were true we'd be able to emulate the behavior somewhat of simpler organisms that don't have the connectivity of a human brain. The fact of the matter is, we simply don't understand cognition. I'd say the challenge in realizing strong AI right now is both a lack of understanding of the brain as well as emulating its mode of computation, which is different from a traditional digital computer.

jacquesm 9y ago

> At this point, processing power rather than theoretical understanding is the limiting factor. No. And you can clearly show this by reasoning like this: if what you say is true then we would be able to simulate a human cognition system but simply slower than real time. The fact that we can't even begin to do that no matter at what speed we would like to set this in motion proves that there is no such thing as sufficient theoretical understanding. To state that even stronger: we are nowhere near close to such understanding. We have roughly the kind of knowledge that you'd be able to glean by taking a hacksaw to a computer, figuring out that there is such a thing as a non-linear element and that you could use this for computation and maybe a basic digital circuit or two. Immensely useful, but not the level of understanding that we would like to have.

dhfhduk 9y ago

I do research in the behavioral sciences. I've published a little on analysis of neural activity, although it's not my area of specialty, but I do specialize in analyses that are cognate to DL models and are often covered in textbooks on DL along with classical DL models. My sense is that the level of understanding of how human cognition works mathematically speaking is roughly similar to our understanding of how DL works in computer science, in the sense that if you asked a cognitive neuroscientist or psychologist how someone classifies cats versus non-cats, you'd get an answer that would seem pretty similar to what you'd get from a computer scientist. The behavioral scientist might go into a lot more detail about certain issues, but that's because the biology is so entertwined. However, I'd also argue that we really don't know much about how human (or any animal) cognition works, and I'd also argue that our understanding of DL is fairly poor, in that a lot of it is tinkering and seeing what happens, without a deep understanding of why it works. There isn't a theory of DL in the same way that there's a Martin-Lof theory of randomness, or a Kolmogorov theory of algorithmic complexity, or a Fisherian model of inference. Also, the sort of tasks currently involved in AI research is a tiny subset of what you encounter in neuroscience and psychology. Most of what is a hot topic in comp sci would basically be classified as perceptual tasks in human behavioral science, maybe at a slightly higher level, and maybe motor control. That leaves things like conscious versus nonconscious processing, reasoning, the role of emotion in decision-making, uncertainty valuation, creativity, etc. etc. etc. I agree that the processing power is an issue but it's only part of the puzzle. One thing that illustrates the complexity of the issues involved, and how we've only begun to scratch the surface, is the article's assertion that comp sci should borrow more the idea of sparse representations from cognitive neuroscience. I thought that was interesting, because in a lot of ways, one of the major trends in the last ten years in human neuroscience is away from this "sparseness" idea. It was a common assumption maybe 15 years ago, but now people routinely get excoriated for invoking that idea. The current paradigm is one more where a lot of pathways/circuits are being recruited simultaneously. Statements like "you might use 10,000 neurons of which 100 are active" would lead to ridicule. The intuitive way of explaining the problem is that even while your brain is trying to decide if you're perceiving a cat versus something else, it's also processing the consequences of that decision along about 10 different dimensions, the implications for the rest of the stimuli coming in, along with a number of other things we just don't understand.