4 ms·
I'm not sure what skip connections are, but I think I have a good guess as to what they are. I've wanted to try a neural network where the output of every laye
by Buttons840 5y ago
I'm not sure what skip connections are, but I think I have a good guess as to what they are.
I've wanted to try a neural network where the output of every layer goes into every subsequent layer. Each layer would thus provide a different perspective to subsequent layers.
Anyone know if this has been tried?
- verdverm 5y agoHave a look at AlphaStar, it's a pretty interesting network of networks that has some skip functionality. The DeepMind lecture series on YouTube is pretty great. You'd likely overdue it with skips everywhere, too many connections to learn and backprop on that learning would likely be difficult
- nomad225 5y agoDensely Connected Convolutional Networks (https://arxiv.org/abs/1608.06993 https://arxiv.org/abs/1608.06993) use the idea you're talking about quite effectively.
- Buttons840 5y agoYeah. That seems like exactly what I was describing. Thanks.