3 ms·
Brain is nothing like attention networks at least in terms of dynamics and individual neuron complexity, my wild ass guess; there is not enough space in DNA to
by neatze 3y ago
Brain is nothing like attention networks at least in terms of dynamics and individual neuron complexity, my wild ass guess; there is not enough space in DNA to store even 10% of brain connections.
What weights are you referring to ?
- BobbyJo 3y agoCluster density, cluster location, eagerness to connect, distribution of cell types, etc. are all coded in DNA and contribute to creating "initial weights". Neuron "weights" need not be stored completely uncompressed. Obviously newborns need to develop and take in stimulus before they "know" anything, so the initial conditions are not sufficient, but they are obviously necessary to make the limited learning useful.
- neatze 3y agoIt always seems naive to compare neural structures and functions to artificial networks, dynamics are simply not there, even single neural would require multi layered network (if not mistaken 7 layer network, and very long training time), it is fascinating how brain self-(re)organizes through out life time from small number of examples here and there. As far my understanding goes to very large degree it is unknown how to model such dynamics, where it would be possible to start with no spiking neurons and evolve effective/stable learning behavior from small number of examples/experiences.
- stevenhuang 3y agoFor all we know it may not even be necessary to model the higher fidelity aspects of our brains for a form of intelligence to emerge. There may be many roads to intelligence. The path biology and evolution took may just be one such path.
- neatze 3y agoI accept this possibility, but because we don't understand, seems wrong to assume any other path, furthermore, there is some evidence that intelligence/cognition is convergent in neural structure and function, in addition to that most ANN are inspired from studying real neurons.
- pmoriarty 3y ago"Cluster density, cluster location, eagerness to connect, distribution of cell types, etc. are all coded in DNA and contribute to creating "initial weights"" Are they coded in DNA, or do epigenetic factors, the environment cells grow in, adjacent cells, their interaction, etc, play a role here as well?
- tomohelix 3y agoThat would depend on what you mean by "connection". The human genome has approximately 200GB of data in it. And this data is highly compressed as it takes 3 different layers of decompression before it gets to usable form.
- le-mark 3y ago> decompression Does this refer to DNA -> MRNA -> to protein or is there some other mechanism here?
- tomohelix 3y agoPost translation modifications are another layer. Also, since you asked, I would like to mention protein interactions with cellular components and tissues. These can be argued as just proteins interacting with each other but the extreme complexity of these higher levels and the unique phenotypes arising from them make me think they are deserved to be treated as another layer of "data decompression".
- fnordpiglet 3y agoA fruit fly has no meaningful learned intelligence yet is more capable than any trained neural net. It’s just a small bundle of nerves laid out according to the model encoded in their DNA. It was trained through millions of years of genetic algorithms, and is fully encoded in the DNA, along with the entire ability to specify the entire physical structure of its body. Why is it surprising our brain has instinctual ability encoded in its structure stemming from our DNA, evolved over the history of life?
- bobbylarrybobby 3y agoNot sure about the 10% point. You could imagine that we got “lucky” and a handful of simple proteins manage to produce the brain (or at least enough of the brain’s connections that the rest can be filled in during infancy via sensory input). Or put another way, maybe evolution optimized for brains that are incredibly adept while only taking a small number of codons (relative to the number of neurons) to describe. The fact that we've managed to l describe the universe through a small set of laws shows that the laws of the universe require much less space to encode than the universe itself, so it's theoretically possible.
- mensetmanusman 3y agoWe have managed to describe a small subset of the universe through a small set of laws. The current incompatibility of gravity and QM and the resulting dark energy/matter explanation means we can’t see or interact with most of the universe.