3 ms·
I don't buy the "one algorithm" idea. I think small changes from region to region can generate completely different dynamics - for example different types of ne
by visarga 8y ago
I don't buy the "one algorithm" idea. I think small changes from region to region can generate completely different dynamics - for example different types of neuron cells, different use of neurotransmitters, or different responses in time. It might look the same under a microscope but hyper-parameters differ between regions. In other words it is a parametrised family of algorithms, not just one.
What would interest me more than the one algorithm is to find out the loss functions that steer our brain into fast learning after birth. We have prior knowledge about the world encoded in the structure of the brain and our loss functions (what the brain optimises for). The brain has many such loss functions - it can't possibly do all it does with just one like neural networks, and they are genetically evolved. These priors could be readily transferred to AI models.
- bra-ket 8y agoThere are no ‘loss functions’, our brain doesn’t optimize in machine learning sense
- visarga 8y agoI think that if it learns, it must have loss functions. We just don't know how they are implemented. If we did, we could replicate the brain priors. They are probably distributed, unlike ML loss functions which distill everything to a single number.
- bra-ket 8y agowell if it learns by remembering sets of features and matching new stimuli against this "database" than we can talk about distance functions in associative memory, and the problem of optimization reduces to a problem of matching (as well as generalizing, i.e constructing abstract representations from simpler ones, as in "chunking"). brain priors would be the distributed representations of stimulus-response pairs stored in memory