25 ms·

So, a neural network of a neural network is just a deeper neural network. The big question in dendritic processing is whether it is used (conflicting informatio

So, a neural network of a neural network is just a deeper neural network. The big question in dendritic processing is whether it is used (conflicting information about that, e.g. Jia&Konnerth's work), whether it represents anything, and how it is learned. Plasticity is all over the place in neurons and takes place also at the dendritic level with cooperation & competition between synapses, temporal dynamics and neuromodulation. The credit assignment problem is hard to solve at the circuit/population level, but dendrites offer an intriguing alternative, as it is possible for them to bidirectionally communicate with the spike initiation site.

svantana 8y ago
Indeed, the Network-in-Network architecture [1] was a compelling idea to get complex activations, until it was realised that it's just a standard neural network which is not fully connected. Since neural networks are universal approximators, it's a bit silly to talk about something else being more powerful, it's all about the prior, bias, and training, which are all subject to the No Free Lunch theorem. [1] https://arxiv.org/abs/1312.4400 https://arxiv.org/abs/1312.4400
- empath75 8y ago
  You can also compose functions together but there’s a reason that programmers don’t generally jam everything into a single function or think about programs that way.
  - deleted 8y ago
    [deleted]
  - svantana 8y ago
    Right, which is one of the main critiques against deep learning, there is no separation of concerns or encapsulation, just a single function matching input to output. But at the end of most days, performance is what matters. Similarly, the brain hardly has a "clean" structure, it's seemingly spaghetti code even though there is some structure to it.