3 ms·
Firstly, (1) and (2) are not rules of thumbs for building nice architectures, but basic properties about supervised learning and what kind of problems it can so
by zodiac 10y ago
Firstly, (1) and (2) are not rules of thumbs for building nice architectures, but basic properties about supervised learning and what kind of problems it can solved.
Even if it were 5 rules of thumb instead of 3, "design an architecture to satisfy 5 rules" is a lot less ad-hoc than "somebody tried lots of things and one seemed to work", which was what I was replying to. We also know why each of these rules are important.
- argonaut 10y agoI'll concede that 1) and 2) are basic properties that an educated layperson would understand after reading a few ML blog posts. Hardly a working theory of neural nets. They're still rules of thumb. You've argued that they are less adhoc than if they didn't exist. That's misses the point: that is still enormously adhoc.