2 ms·
> What’s the significance of this paper? We're progressing from static neural nets (like the old TensorFlow) to dynamic nets (like PyTorch) and even dynamicall
by visarga 9y ago
> What’s the significance of this paper?
We're progressing from static neural nets (like the old TensorFlow) to dynamic nets (like PyTorch) and even dynamically generated on-the-fly neural nets (hypernets) which is great for creating architectures that better fit the problem, case by case.
This paper shows another way to generate a net from a net, but this time with self replication. Self replication could be useful for evolutionary optimisation, and, as they say in the paper, could become a rudimentary form of introspection. The idea of a "self-replication loss term" is very exciting, it's rare that we see such innovation in loss functions.