4 ms·
>The mainstream deep learning academic community welcomes theoretical work but articles on new techniques which beat SOTA are given much more attention than art
by colllectorof 11y ago
>The mainstream deep learning academic community welcomes theoretical work but articles on new techniques which beat SOTA are given much more attention than articles on setting up a theoretical structure of the models.
The fact that beating an accuracy test on some arbitrary data set is seen as more important than understanding why and when your methods works (or do not work) is deeply disturbing.
- ogrisel 11y agoIt's not an arbitrary dataset. To be taken seriously you need to beat SOTA on several standard benchmark datasets where only reaching SOTA on one is already hard. Or alternatively, reach comparable level of validation accuracy with significantly less compute or memory usage (computational complexity). Or alternatively, reach comparable level of validation accuracy with only a subset of labeled samples + unlabeled samples (sample complexity).
- eli_gottlieb 11y agoThat still amounts to publishing when you can say, "We know it does work" as opposed to "We know why it works".
- aminorex 11y ago...which is very valuable. After all, we don't know when, if ever, a satisfactory explanation (by what criteria?) will derive. To defer all progress in the field until it should meet the standards of some other field would be a great loss, and might preclude any number of future further discoveries.
- eli_gottlieb 11y agoAs far as I understand the concept of "science", an empirical observation (including "it works from an engineering perspective, kinda") is only ever an observation. Scientific progress, by definition, involves advancing our understanding of hows and whys.