3 ms·
This seems quite intuitive to me: When you have nothing to learn, you need to memorize the data. But when there is structure, it is easier to memorize the stru
by elcomet 8y ago
This seems quite intuitive to me:
When you have nothing to learn, you need to memorize the data. But when there is structure, it is easier to memorize the structure, so the network will learn this first (and will memorize after).
- srean 8y agoBut thats the crux of the question: why and how does it not just memorize when we know it can do so easily ?
- elcomet 8y agoMaybe just because it's easier to find patterns than to memorize (if you have a lot of data).
- currymj 8y agoThat sounds like it’s probably right to me. But so do lots of things that turn out to be wrong. I wish we had a better grasp of what is happening, not just plausible stories. I’m already sick of doing alchemical tinkering to find a model that works.
- elcomet 8y agoYeah, I agree it would be nice to have some theoretical guarantees on the architecture we need based on the problem, and the size of the data