3 ms·
Wait a second. I am missing something. The definition of science is: sci·ence /ˈsīəns/ noun the intellectual and practical activity encompassing the systematic
by fabmilo 6y ago
Wait a second. I am missing something. The definition of science is:
sci·ence
/ˈsīəns/
noun
the intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment.
I might be bias (as I write such GPU rigs), but from I don't see how is not a scientific process trying different architecture configurations.
I like the Sturgeon's Law reference from this thread.
- xapata 6y agoI'd rephrase the critique to say that you're doing descriptive science. It's like a naturalist drawing pictures of birds. The most interesting work comes later, when someone suggests a theory to explain it all.
- sdenton4 6y agoThere's lots of theories! It's just that there's not really sufficient mathematics available to determine which ones are provably correct. (My favorite example of this is the seemingly never-ending arguments over the REAL reason that batch normalization works...) And there are real advances happening because of advances on the theory side, as well: I would say that ResNet is an excellent example of this, bringing some insights from differential equations into model architecture, and greatly advancing the quality of classification. All that said, I do worry that 'fundamental science' advances might get overlooked if they don't contribute to new high scores. For example, if you can prove that a certain model will 'work' (for some value of 'work'), the model may be hobbled by the need to prove things, and thus not competitive... In which case, the proof techniques might be lost in the howling void of the arxiv.