4 ms·
We should train another neural network to produce the initial values for the target network. Training data would be the initial values of the sub network along
by plutonorm 7y ago
We should train another neural network to produce the initial values for the target network. Training data would be the initial values of the sub network along with its loss. Then we have the network generate initial values that are more likely to lead yo useful sub networks.
- intuitionist 7y agoThis is essentially the idea behind “meta-learning,” or “learning to learn,” with the slight difference that most of the meta-learning literature aims to initialize networks that can learn quickly (“few-shot learning”). It has pretty good theoretical grounding but in practice seems to be quite expensive.
- plutonorm 7y agoLol. I've seen idea after idea, dismissed and misunderstood on Internet forums, from hackernews to reddit.com/r/machine learning. Time after time I see ideas, my own and those of others, that were dismissed, spawning papers by researchers. Such a toxic atmosphere, I don't know why I bother posting. I am well aware of what meta learning is. If you think that because the idea above comes under the umbrella of metalearning it isn't an interesting idea, then I don't know what to say to you... and the fact that it's downvoted just goes to show the lack of creativity and intuition exhibited by your average hackernews member.
- intuitionist 7y agoI never implied that it wasn’t an interesting idea, and your weird defensiveness suggests to me that there might be a common thread to your getting repeatedly dismissed/downvoted other than misunderstood genius.