3 ms·
Genuinely shocked by this question. There is no part of the data science stack that is in any way settled. Even more broadly I don’t think statistical best prac
by thom 6y ago
Genuinely shocked by this question. There is no part of the data science stack that is in any way settled. Even more broadly I don’t think statistical best practice is at all established, or at least well distributed.
You haven’t even missed the boat in terms of being able to make money off raw buzzwords and zero skills.
At the very basic technical level, there’s infinite work to be done optimising machine learning systems. This includes not just the fashionable issues of faster more accurate (or even less accurate in terms of floating point!) deep learning, but also moving Bayesian approaches like MCMC to multiple cores and GPUs.
There’s infinite work to be done on finding the right topology for a machine learning system. This applies not just to neural network layers but also to traditional stats (i.e. multiverse analysis).
There’s infinite work to be doing in understanding, cleaning and preparing datasets. Something as clunky as tidyverse can’t be close to the final form here. We’ve only just started talking about feature stores etc.
There’s infinite work to be done improving notebooks, integrating better software engineering practise into the workflow but also in terms of productionisation of models created therein.
All this is just platform stuff as well. It doesn’t even touch on the fact that businesses everywhere are terrible at formulating questions to be answered by stats, terrible at communicating those answers and terrible at even knowing this is a valuable endeavour in the first place.
I cannot imagine a boat harder to miss.