2 ms·
We used to call this model fishing and regard models that came from places that did it with fear and suspicion (due to sudden failure and poor performance in pr
by sgt101 8y ago
We used to call this model fishing and regard models that came from places that did it with fear and suspicion (due to sudden failure and poor performance in production)
What has changed that people think this is a wise approach?
- bitL 8y agoAutoML and automated architecture/hyperparameter search used by FAANG, producing things like AmoebaNet-D, that beat all previous state-of-art results.
- matt4077 8y agoThey don't suddenly fail any more, and performance in production is fine... This is a rather empirical science right now. If forced to speculate, I'd say far larger datasets are part of the answer. If you can afford to hold back 30%+ of your data for (multi-tranche) verification the difference between testing and production becomes a problem for the philosophers.
- sgt101 8y agoHmm this isn't my experience. My experience is that highly complex models tend to reflect accurately the ground truth in the current dataset, but not the domain theory that generated it, so when things change (like, prevailing weather conditions, political alignments, interest rates, data usage) which move the distribution (but not the domain theory) they do fail. The question is : are you trading poorer compression for more fidelity or are you learning the system that generates the data?