4 ms·

It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statist

It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random forests and the stats-founded but CS-led approaches are very general models. That leaves statisticians a big opening because a more specific model can be chosen to obtain more accurate predictions. These days I'm positioning myself to clean-up the messes / save broken ML models. Turns out [stats] theory is very practical. :-)

Eridrus 8y ago
I'm curious what resources you found useful to learn stats modelling and what sorts of approaches have been useful. On one hand, it's almost a tautoloy that specific models should be better than general models, but I worked on some 2d time series classification with a statistician and afterwards, for kicks, I replaced the entire thing with a CNN+LSTM and it worked just as well as the whole complicated model he had come up with.
- RA_Fisher 8y ago
  I highly recommend this econometrics text for getting started with statistics: https://www.amazon.com/Principles-Econometrics-5th-Carter-Hill-ebook/dp/B07BNZ6MWM https://www.amazon.com/Principles-Econometrics-5th-Carter-Hi... For modeling I found Wooldridge's panel and cross-section data book very useful: https://www.amazon.com/Econometric-Analysis-Cross-Section-Panel/dp/0262232588 https://www.amazon.com/Econometric-Analysis-Cross-Section-Pa... Greene is a really useful reference text: https://www.amazon.com/Econometric-Analysis-8th-William-Greene/dp/0134461363 https://www.amazon.com/Econometric-Analysis-8th-William-Gree... For advanced stats theory, I recommend Casella and Berger https://www.amazon.com/Statistical-Inference-George-Casella/dp/0534243126 https://www.amazon.com/Statistical-Inference-George-Casella/... Hope that helps! The more specific a model can be made to the problem at hand, the better it'll perform. Supervised ML models are great starting / baseline models.
  - tomrod 8y ago
    I second Wooldridge. Greene I found to be much denser without providing much additional insight. It is a popular MS/PhD entry text though. I add any of Ken Train's work to this mix, especially on estimating discrete choice theory.
curiousgal 8y ago
Because saying "relevant username" is frowned upon I'll just point out that R A Fisher is "a genius who almost single-handedly created the foundations for modern statistical science"[0] 0.https://en.m.wikipedia.org/wiki/Ronald_Fisher https://en.m.wikipedia.org/wiki/Ronald_Fisher
- srean 8y ago
  ... and someone who would be very difficult to "out asshole" or to out do in male chauvinism. Those are criticism on personality, on the technical side it took the community a long time to undo the damage of promoting non robust parametric statistics. But this much is certain he pulled statistics into the realms of math -- no mean feat.
  - RA_Fisher 8y ago
    That's true and it's well documented. I think E.T. Jaynes gives a poigntent reflection in his Theory of Probability.
- mlthoughts2018 8y ago
  It’s funny to me, as a professional statistician, because most methods popularized by Fischer et al in the early 1900s are wildly inappropriate for practical problems, especially policy decision science or causal inference. All the theory behind t-testing, Wald testing, using the detivatives of the log likelihood near to the MLE point estimate in order to also estimate standard errors when no analytical solution exists, ANOVA, instrumental variables, etc. It is in no sense exaggerative or incendiary to say that whole collection of stuff is truly garbage statistics that is insanely rife with counter-intuitive results, common situations when minor violations of the assumptions can easily lead to statistically significant results of the wrong sign, and common practical needs (like model selection without doing a bunch of pairwise or subset selection calculations, or correcting for multicollinearity in large regressions where calculating something like variance inflation factors is totally intractable) are difficult or impossible. Modern Bayesian approaches fully and entirely subsume these techniques, and not just for large data (in fact, using Bayesian methods is more critical for small data), and also not because of modern computing frameworks, but because, from very first principle of null-hypothesis significance testing, that whole field of stats/econometrics is fundamentally incapable of giving evidence or estimations that could address the very questions that the whole field is based on. NHST basically solves a type of inference problem that nobody can ever actually have in reality, and which is almost always not even approximately close enough to actually be non-misleading. NHST is like the stats analogue of Javascript: a horrible historical accident that gained market traction despite being utterly and unequivocally a bad choice for the very problem domain it’s intended to be used for. The historical accident of adoption and momentum in Javascript sets back professional computer science by decades until it’s eventually wholesale replaced with something whose first principles are actually appropriate. That same reckoning is in flux in many fields of statistics, as the fundamental unreliability of NHST estimation is more understood and drop-in Bayesian replacements are more available.
cs702 8y ago
True. On the other hand, the "more ignorant CS approach" has produced impressive achievements in language tasks (e.g., translation), visual tasks (e.g., image generation), game playing tasks (e.g., Go), agent-in-virtual-world tasks (e.g., DOTA), and robot-in-real-world tasks (e.g., self-driving cars). Academic statistics departments often seem to be "20 years behind" on all those fronts...
- md2be 8y ago
  No doubt that club of statistical significance held back many statisticians.
thanatropism 8y ago
I know a handful of Econ phds working in data science; and Google, FB etc. have hired top economists as well. The Phineas Gage of applied quantitative Econ is demand estimation. You typically want to know the elasticity of quantities sold to price so to inform pricing policies. But the problem is that causality is cloudy -- low prices cause a decrease in supply -- so you never know what you're looking at. People with a decent training in econometrics know how to treat this problem. I'm pretty sure orgs like Amazon were trying to do naive demand estimation, fell flat on their noses and copped to having to hire people who have thought about the underlying conceptual issues before.

Eridrus 8y ago

I'm curious what resources you found useful to learn stats modelling and what sorts of approaches have been useful. On one hand, it's almost a tautoloy that specific models should be better than general models, but I worked on some 2d time series classification with a statistician and afterwards, for kicks, I replaced the entire thing with a CNN+LSTM and it worked just as well as the whole complicated model he had come up with.

RA_Fisher 8y ago
I highly recommend this econometrics text for getting started with statistics: https://www.amazon.com/Principles-Econometrics-5th-Carter-Hill-ebook/dp/B07BNZ6MWM https://www.amazon.com/Principles-Econometrics-5th-Carter-Hi... For modeling I found Wooldridge's panel and cross-section data book very useful: https://www.amazon.com/Econometric-Analysis-Cross-Section-Panel/dp/0262232588 https://www.amazon.com/Econometric-Analysis-Cross-Section-Pa... Greene is a really useful reference text: https://www.amazon.com/Econometric-Analysis-8th-William-Greene/dp/0134461363 https://www.amazon.com/Econometric-Analysis-8th-William-Gree... For advanced stats theory, I recommend Casella and Berger https://www.amazon.com/Statistical-Inference-George-Casella/dp/0534243126 https://www.amazon.com/Statistical-Inference-George-Casella/... Hope that helps! The more specific a model can be made to the problem at hand, the better it'll perform. Supervised ML models are great starting / baseline models.
- tomrod 8y ago
  I second Wooldridge. Greene I found to be much denser without providing much additional insight. It is a popular MS/PhD entry text though. I add any of Ken Train's work to this mix, especially on estimating discrete choice theory.

curiousgal 8y ago

Because saying "relevant username" is frowned upon I'll just point out that R A Fisher is "a genius who almost single-handedly created the foundations for modern statistical science"[0] 0.https://en.m.wikipedia.org/wiki/Ronald_Fisher https://en.m.wikipedia.org/wiki/Ronald_Fisher

srean 8y ago
... and someone who would be very difficult to "out asshole" or to out do in male chauvinism. Those are criticism on personality, on the technical side it took the community a long time to undo the damage of promoting non robust parametric statistics. But this much is certain he pulled statistics into the realms of math -- no mean feat.
- RA_Fisher 8y ago
  That's true and it's well documented. I think E.T. Jaynes gives a poigntent reflection in his Theory of Probability.
mlthoughts2018 8y ago
It’s funny to me, as a professional statistician, because most methods popularized by Fischer et al in the early 1900s are wildly inappropriate for practical problems, especially policy decision science or causal inference. All the theory behind t-testing, Wald testing, using the detivatives of the log likelihood near to the MLE point estimate in order to also estimate standard errors when no analytical solution exists, ANOVA, instrumental variables, etc. It is in no sense exaggerative or incendiary to say that whole collection of stuff is truly garbage statistics that is insanely rife with counter-intuitive results, common situations when minor violations of the assumptions can easily lead to statistically significant results of the wrong sign, and common practical needs (like model selection without doing a bunch of pairwise or subset selection calculations, or correcting for multicollinearity in large regressions where calculating something like variance inflation factors is totally intractable) are difficult or impossible. Modern Bayesian approaches fully and entirely subsume these techniques, and not just for large data (in fact, using Bayesian methods is more critical for small data), and also not because of modern computing frameworks, but because, from very first principle of null-hypothesis significance testing, that whole field of stats/econometrics is fundamentally incapable of giving evidence or estimations that could address the very questions that the whole field is based on. NHST basically solves a type of inference problem that nobody can ever actually have in reality, and which is almost always not even approximately close enough to actually be non-misleading. NHST is like the stats analogue of Javascript: a horrible historical accident that gained market traction despite being utterly and unequivocally a bad choice for the very problem domain it’s intended to be used for. The historical accident of adoption and momentum in Javascript sets back professional computer science by decades until it’s eventually wholesale replaced with something whose first principles are actually appropriate. That same reckoning is in flux in many fields of statistics, as the fundamental unreliability of NHST estimation is more understood and drop-in Bayesian replacements are more available.

cs702 8y ago

True. On the other hand, the "more ignorant CS approach" has produced impressive achievements in language tasks (e.g., translation), visual tasks (e.g., image generation), game playing tasks (e.g., Go), agent-in-virtual-world tasks (e.g., DOTA), and robot-in-real-world tasks (e.g., self-driving cars). Academic statistics departments often seem to be "20 years behind" on all those fronts...

md2be 8y ago
No doubt that club of statistical significance held back many statisticians.

thanatropism 8y ago

I know a handful of Econ phds working in data science; and Google, FB etc. have hired top economists as well. The Phineas Gage of applied quantitative Econ is demand estimation. You typically want to know the elasticity of quantities sold to price so to inform pricing policies. But the problem is that causality is cloudy -- low prices cause a decrease in supply -- so you never know what you're looking at. People with a decent training in econometrics know how to treat this problem. I'm pretty sure orgs like Amazon were trying to do naive demand estimation, fell flat on their noses and copped to having to hire people who have thought about the underlying conceptual issues before.