4 ms·
That is honestly a much more interesting result than the title would suggest. We know the brain can't do backprop (neurons are one way), but the fact that there
by ravi-delia 3y ago
That is honestly a much more interesting result than the title would suggest. We know the brain can't do backprop (neurons are one way), but the fact that there is convergence in algorithm is very fun.
- monocasa 3y ago> neurons are one way Synapses are one way. Biological neurons have many synapses.
- ravi-delia 3y agoTechnically true! But one, very few neurons have connections going both ways to each other- any two neurons will be lined up axon to synapse one way or the other. And two, the specific structures that compose most of the cortex are remarkably unidirectional. The neocortex consists mostly of little chunks of neurons arranged in layers, themselves hooked up snout to tail. There are loops, eventually, but only on the macro scale. Outside the cortex, and even moreso outside the brain, there is a far greater diversity of structures. Two neurons plugged into each other form an excellent basis for a timer, damper, clock, or even primitive stimulus response. There are structures taking proprioceptive info from the joints and performing integrals on them. It's neat because the circuits are so small we can actually understand them completely! But within the brain things definitely are not laid out right for backprop.
- ckemere 3y agoI suppose. Equivalent results about natural images and edge detection have been reported in the image processing (classical, not deep) ML literature 20 years ago...
- ravi-delia 3y agoOh yeah for sure. But as neat as edge detection in the eye is, it isn't learned. I'm enjoying seeing similar results in highly unstructured models like transformers.