4 ms·
Indeed, the Network-in-Network architecture [1] was a compelling idea to get complex activations, until it was realised that it's just a standard neural network
by svantana 8y ago
Indeed, the Network-in-Network architecture [1] was a compelling idea to get complex activations, until it was realised that it's just a standard neural network which is not fully connected. Since neural networks are universal approximators, it's a bit silly to talk about something else being more powerful, it's all about the prior, bias, and training, which are all subject to the No Free Lunch theorem.
[1] https://arxiv.org/abs/1312.4400 https://arxiv.org/abs/1312.4400
- empath75 8y agoYou can also compose functions together but there’s a reason that programmers don’t generally jam everything into a single function or think about programs that way.
- deleted 8y ago[deleted]
- svantana 8y agoRight, which is one of the main critiques against deep learning, there is no separation of concerns or encapsulation, just a single function matching input to output. But at the end of most days, performance is what matters. Similarly, the brain hardly has a "clean" structure, it's seemingly spaghetti code even though there is some structure to it.