4 ms·
I'd rephrase the critique to say that you're doing descriptive science. It's like a naturalist drawing pictures of birds. The most interesting work comes later,
by xapata 6y ago
I'd rephrase the critique to say that you're doing descriptive science. It's like a naturalist drawing pictures of birds. The most interesting work comes later, when someone suggests a theory to explain it all.
- sdenton4 6y agoThere's lots of theories! It's just that there's not really sufficient mathematics available to determine which ones are provably correct. (My favorite example of this is the seemingly never-ending arguments over the REAL reason that batch normalization works...) And there are real advances happening because of advances on the theory side, as well: I would say that ResNet is an excellent example of this, bringing some insights from differential equations into model architecture, and greatly advancing the quality of classification. All that said, I do worry that 'fundamental science' advances might get overlooked if they don't contribute to new high scores. For example, if you can prove that a certain model will 'work' (for some value of 'work'), the model may be hobbled by the need to prove things, and thus not competitive... In which case, the proof techniques might be lost in the howling void of the arxiv.