4 ms·
This supports the idea that, increasingly, machine learning is coming back full circle to support neuroscience. Previously, AI researchers looked at the brain f
by eboyjr 8y ago
This supports the idea that, increasingly, machine learning is coming back full circle to support neuroscience. Previously, AI researchers looked at the brain for inspiration and now more than ever neuroscientists are being inspired by advances in deep learning, etc.
- make3 8y agothis would be the dream, but biological neural networks don't do gradient decent, so have little to do with ANNs until we discover how they really train.
- jacquesm 8y agoIt is perfectly possible to build neural networks with other functions besides gradient descent.
- make3 8y agoI agree. I'm saying we don't know how biological neural networks are trained, which is a pretty important part of any learning construct, so we can't really know how actually similar artificial neural nets are.
- taneq 8y agoWhat other methods are there for training a neural network? Even blind evolutionary algorithms (assuming each generation is initialized to be somewhat 'near' or 'similar' to its progenitors) are effectively descending a gradient.
- luk32 8y agoTo my knowledge they explore solution space but not through a gradient descent but by probing. How can you perform a GD on non continuous function? Genetic algorithms and evolutionary methods can explore disjoint domains and functions hard from analytical pov. Just because they can be used for gradient descend doesn't mean they use it.
- akyu 8y agoIt would be remarkable if biological neural networks don't use gradient descent. They very likely do even if it is a highly approximate gradient. Neurons are optimizing for some objective, and the gradient is usually a very efficient way to do that optimization. It would be remarkable if evolution missed out on such a thing. Even simple Hebbian learning can be viewed as a gradient descent technique. Gradients can be calculated in remarkably simple ways. Look at Hinton's recirculation networks for example. I think you meant to say backprop.
- gnulinux 8y agoHow would brain do gradient descent without backpropagation?
- eli_gottlieb 8y agoMicrocircuits could descend the gradient of a function they can evaluate locally.
- yosito 8y agoI don't doubt that there are many similarities between how machine learning works and how brains work, but this seems like a pretty myopic trend of confirmation bias. Neurons and brains are so much more complex than machine learning and it will be really unfortunate if we limit ourselves to the machine learning model in neuroscience.
- 2bitencryption 8y agomaybe the functional nodes (wetware neurons vs software neurons) are very different, but it seems like the way they are manipulated via back-propagation, pooling, recurrence, layerings, etc, are similar, right? because, at the end of the day, it's more about how behavior is emergent than how behavior functions physically, I would say. I would guess that if we ever get to some true sci-fi AI "consciousness", it would just be a hyper-scaled version of what we already have. But that's just fun speculation.
- tehsauce 8y agoThis is not correct, artificial NNs are not functionally related to neurons in the brain. Nothing like back-propagation has been observed in real neurons. The early layers of a CNN may be similar to early perceptrons in the brain, but beyond this any connection between the two is fantasy.
- Retric 8y agoNope. Brains don't use layers or back propagation. CS NN's are really just a cool name for math that has almost nothing in common with how the brain functions. There is a lot of misinformation about how the brain works. For example you see a lot of drawings with different parts of the brain doing different things. However, if you look at an actual brain almost none of this is physically obvious. At best those diagrams show what stops working when that part is damaged, though again plenty of people have very different structures and it still mostly works.
- yosito 8y agoHonestly, this is more like having invented solar cells and then saying that that's how trees work. Turning light into energy is just a small part of what trees do, and solar cells are only an approximation of the idea but the actual implementation is very very different from photosynthesis.