5 ms·

It's not like that hasn't been tried before though. Boltzmann machines have all-to-all connectivity and RNNs have feedback connections (and sometimes backward c

• ad404b8a372f2b9 6y ago
  Convolutional neural networks were also tried and failed to fulfill their potential for 20 years (or 50 depending on how far you go back) until they didn't and now they're ubiquitous. You cannot discard a whole area of research because one or some implementations of a concept have so far failed to become competitive. All to all connectivity is possibly the worst paradigm unless you want to do architecture search so I wouldn't hold it as an example of anything wrt this concept. As for recurrent neural networks to my knowledge they've only been used for recurrence in time or space not recurrence between layers for a single sample, though I might be wrong, so they're not relevant to what I'm speaking of. Though there is some work on skip connections (forward and backward) which takes some inspiration from their gating mechanisms.