7 ms·

The most important statistical ideas of the past 50 years

173 points by BerislavLopac 6y ago

madars 6y ago
If you are looking for native PDF interface, the paper appears to be https://arxiv.org/abs/2012.00174 https://arxiv.org/abs/2012.00174
- bambax 6y ago
  Ah thanks! (This non-zoomable format with random comments that add no value is extremely annoying... What are people thinking when they do this.)
sedeki 6y ago
(Pretty much) previous HN discussion: https://news.ycombinator.com/item?id=26374788 https://news.ycombinator.com/item?id=26374788
RcouF1uZ4gsC 6y ago
Looking at statistics as it is used in published papers overall, it seems the the most important statistical idea of the last 50 years is p-hacking. Being able to do multiple analysis of the same data, and find one where p < 0.05 has revolutionized academia. Entire careers have been built around p-hacking. Of all the statistical idea, it is the one which arguably has had the most influence in the life of the academician.
- glgoose 6y ago
  If you torture the data long enough, it will confess. ~Ronald Coase
- eeegnu 6y ago
  What kind of solution is there to this? Only trust studies which have been successfully reproduced several times? Raise the bar for an acceptable p value so it becomes less practical to p-hack?
  - glgoose 6y ago
    I think a lot of publications and scientists are now aware of this so they adjust their experimental design to not allow for a whole bunch of parameters or small sample/population size.
  - nextos 6y ago
    Focusing on effect sizes! It's much much harder to lie with effect sizes. For instance, in genetics there are entire careers built on an association of certain genetic polymorphism with a particular phenotype. Even if the p-value can be reproduced with huge sample sizes, the effect size of the association is usually tiny. In other words, if you carry the risk allele the odds of developing a disease are usually only slightly bigger than if you didn't. But this is often not told to funders! Focusing on effect sizes would drive research effort to things that actually make a difference. You can do effect sizes with something as simple as a confidence interval, or with a fully fledged Bayesian model that tries to estimate the posterior distribution of some parameter.
    - kkoncevicius 6y ago
      Modern genome association studies rarely look for single allele effects anymre. The current views shift towards polygenic or sometimes even omnigenic (meaming majority of genes can have indirect effects on the phenotype of disease) associations. No one expects to see big effect sizes because very few people will have differences in any one particular loci. So under these models simple effect sizes are less important than p-values.
    - dumb1224 6y ago
      I think effect size is a very useful tool if you know some odds a priori. E.g if you can compare the effect size with some known features/variables. It can then give you a rough feeling how true it might be. If the features are small as genetic variants then people will have a hard time estimating the supposed effect (unless it's a functional one that alters the encoded protein e.g).
  - laGrenouille 6y ago
    Preregistration of studies [0]. Submit design and models (e.g., logistic regression with these covariates and these exclusion criteria) before collecting data. This is relatively easy to implement and hard to get around, but does need to be paired with more acceptance of the importance of null results in advancing science. [0] https://en.wikipedia.org/wiki/Preregistration_(science) https://en.wikipedia.org/wiki/Preregistration_(science)
- ellisv 6y ago
  Most of published statistics are not performed by statisticians.
- hannob 6y ago
  Counterpoint: Pretty sure p-hacking is older than 50 years. Yeah, the term only came up recently. But ultimately p-hacking is "just" exploiting freedom in the possible interpretation of statistical results. Also it wasn't really "invented", it's just what humans do if they have data in front of them and some expectation what the data should tell them. I'm pretty sure the first p-hacker was the first person to do any form of early statistics. I guess the greatest invention would be a method to prevent p-hacking. (And yeah, I know about preregistration. It doesn't work the way it's done right now, because a) most people don't do it, b) preregistration is rarely precise enough to really prevent p-hacking and c) nobody cares if you switch your outcomes.)
- CuriouslyC 6y ago
  Technically, P-hacking isn't just ethically wrong, it's also statistically incorrect. If you're testing the same data for multiple hypotheses, or performing the same test on multiple data sets, you need to perform a Bonferroni correction for multiple hypothesis testing.
worik 6y ago
I knoew when I saw that title it would trigger me! Sure did: Number two, the Bootstrap. It is the "Stationary Bootstrap" because it can only be used on stationary data (for very obvious reasons). Problem is that tests for stationarity are very poor. I have seen it used in academic finance to prove technical analysis. Yes, really (A clue to the clueless - except in very special circumstances technical analysis does not work). Completely inappropriate to bootstrap financial returns data as they are not stationary. I am talking about White's Reality Check: which was devised by Herbert White who was a Distinguished Professor of Economics at the University of California. It is completely bananas, (final return data is not stationary. Bootstrapping it gives random noise. Duh!). When I looked at it about decade ago the paper it was developed in (two papers actually) were among the most cited in the field. I am not convinced that it can be used in stationary data, though I only have experience debunking it, not making good use of it. If something seems too good to be true.... And the bootstrap promises to take your too small number of data and make it into a lot of data. I see the paper here did not bother with stationarity. I stopped reading there. I cannot bear the myths and self deception statisticians tell themselves. I did it as a graduate student for far too long.
- ellisv 6y ago
  Seems like an oddly specific critique. The paper here doesn’t concern itself with timeseries specifically (nor spatial, spatiotemporal, etc).
- clircle 6y ago
  No halfway decent statistician has ever advised that one bootstrap a time series. Naive bootstrap obviously does not apply to that data. And the bootstrap is not for taking a small amount of data and making it a large amount, it’s an empirical estimate of a sampling distribution.
  - worik 6y ago
    What was Herbert White then?
    - 6y ago
sinuhe69 6y ago
Why can they not present such a short notes (without graphics and tables) in a more accessible format like HTML is beyond me!
- hansvm 6y ago
  They probably can but simply haven't because they haven't thought about the issue at all, aren't willing to put in the extra work (the PDF version was probably all but mandatory if they're from academia), didn't want to put in the significant amount of extra work to make an HTML paper look halfway decent, didn't want to learn a new set of tools right now when their PDF toolchain works just fine, .... More importantly, they're people and probably had _some_ reason for doing what they did. If we'd prefer an HTML version instead or in addition, we ought to make an honest effort to understand their perspective and address their concerns rather than just bemoan their choices.
  - neolog 6y ago
    PDF is more stable over time.
johndoe42377 6y ago
Statistics (merely observation and counting) cannot establish or even discover causation in principle. Period. No matter how many per reviewed gibberish papers will be published, the philosophical principle will stand. What you observe are effects. Causes are not there. Only discrete, fully observable, simple systems, like dice or a deck of cards, could be modelled adequately. Most of real world complex systems with multiple causation cannot.

173 points by BerislavLopac 6y ago

If you are looking for native PDF interface, the paper appears to be https://arxiv.org/abs/2012.00174 https://arxiv.org/abs/2012.00174

bambax 6y ago
Ah thanks! (This non-zoomable format with random comments that add no value is extremely annoying... What are people thinking when they do this.)

sedeki 6y ago

(Pretty much) previous HN discussion: https://news.ycombinator.com/item?id=26374788 https://news.ycombinator.com/item?id=26374788

RcouF1uZ4gsC 6y ago

Looking at statistics as it is used in published papers overall, it seems the the most important statistical idea of the last 50 years is p-hacking. Being able to do multiple analysis of the same data, and find one where p < 0.05 has revolutionized academia. Entire careers have been built around p-hacking. Of all the statistical idea, it is the one which arguably has had the most influence in the life of the academician.

glgoose 6y ago
If you torture the data long enough, it will confess. ~Ronald Coase
eeegnu 6y ago
What kind of solution is there to this? Only trust studies which have been successfully reproduced several times? Raise the bar for an acceptable p value so it becomes less practical to p-hack?
- glgoose 6y ago
  I think a lot of publications and scientists are now aware of this so they adjust their experimental design to not allow for a whole bunch of parameters or small sample/population size.
- nextos 6y ago
  Focusing on effect sizes! It's much much harder to lie with effect sizes. For instance, in genetics there are entire careers built on an association of certain genetic polymorphism with a particular phenotype. Even if the p-value can be reproduced with huge sample sizes, the effect size of the association is usually tiny. In other words, if you carry the risk allele the odds of developing a disease are usually only slightly bigger than if you didn't. But this is often not told to funders! Focusing on effect sizes would drive research effort to things that actually make a difference. You can do effect sizes with something as simple as a confidence interval, or with a fully fledged Bayesian model that tries to estimate the posterior distribution of some parameter.
  - kkoncevicius 6y ago
    Modern genome association studies rarely look for single allele effects anymre. The current views shift towards polygenic or sometimes even omnigenic (meaming majority of genes can have indirect effects on the phenotype of disease) associations. No one expects to see big effect sizes because very few people will have differences in any one particular loci. So under these models simple effect sizes are less important than p-values.
  - dumb1224 6y ago
    I think effect size is a very useful tool if you know some odds a priori. E.g if you can compare the effect size with some known features/variables. It can then give you a rough feeling how true it might be. If the features are small as genetic variants then people will have a hard time estimating the supposed effect (unless it's a functional one that alters the encoded protein e.g).
- laGrenouille 6y ago
  Preregistration of studies [0]. Submit design and models (e.g., logistic regression with these covariates and these exclusion criteria) before collecting data. This is relatively easy to implement and hard to get around, but does need to be paired with more acceptance of the importance of null results in advancing science. [0] https://en.wikipedia.org/wiki/Preregistration_(science) https://en.wikipedia.org/wiki/Preregistration_(science)
ellisv 6y ago
Most of published statistics are not performed by statisticians.
hannob 6y ago
Counterpoint: Pretty sure p-hacking is older than 50 years. Yeah, the term only came up recently. But ultimately p-hacking is "just" exploiting freedom in the possible interpretation of statistical results. Also it wasn't really "invented", it's just what humans do if they have data in front of them and some expectation what the data should tell them. I'm pretty sure the first p-hacker was the first person to do any form of early statistics. I guess the greatest invention would be a method to prevent p-hacking. (And yeah, I know about preregistration. It doesn't work the way it's done right now, because a) most people don't do it, b) preregistration is rarely precise enough to really prevent p-hacking and c) nobody cares if you switch your outcomes.)
CuriouslyC 6y ago
Technically, P-hacking isn't just ethically wrong, it's also statistically incorrect. If you're testing the same data for multiple hypotheses, or performing the same test on multiple data sets, you need to perform a Bonferroni correction for multiple hypothesis testing.

worik 6y ago

I knoew when I saw that title it would trigger me! Sure did: Number two, the Bootstrap. It is the "Stationary Bootstrap" because it can only be used on stationary data (for very obvious reasons). Problem is that tests for stationarity are very poor. I have seen it used in academic finance to prove technical analysis. Yes, really (A clue to the clueless - except in very special circumstances technical analysis does not work). Completely inappropriate to bootstrap financial returns data as they are not stationary. I am talking about White's Reality Check: which was devised by Herbert White who was a Distinguished Professor of Economics at the University of California. It is completely bananas, (final return data is not stationary. Bootstrapping it gives random noise. Duh!). When I looked at it about decade ago the paper it was developed in (two papers actually) were among the most cited in the field. I am not convinced that it can be used in stationary data, though I only have experience debunking it, not making good use of it. If something seems too good to be true.... And the bootstrap promises to take your too small number of data and make it into a lot of data. I see the paper here did not bother with stationarity. I stopped reading there. I cannot bear the myths and self deception statisticians tell themselves. I did it as a graduate student for far too long.

ellisv 6y ago
Seems like an oddly specific critique. The paper here doesn’t concern itself with timeseries specifically (nor spatial, spatiotemporal, etc).
clircle 6y ago
No halfway decent statistician has ever advised that one bootstrap a time series. Naive bootstrap obviously does not apply to that data. And the bootstrap is not for taking a small amount of data and making it a large amount, it’s an empirical estimate of a sampling distribution.
- worik 6y ago
  What was Herbert White then?
  - 6y ago

sinuhe69 6y ago

Why can they not present such a short notes (without graphics and tables) in a more accessible format like HTML is beyond me!

hansvm 6y ago
They probably can but simply haven't because they haven't thought about the issue at all, aren't willing to put in the extra work (the PDF version was probably all but mandatory if they're from academia), didn't want to put in the significant amount of extra work to make an HTML paper look halfway decent, didn't want to learn a new set of tools right now when their PDF toolchain works just fine, .... More importantly, they're people and probably had _some_ reason for doing what they did. If we'd prefer an HTML version instead or in addition, we ought to make an honest effort to understand their perspective and address their concerns rather than just bemoan their choices.
- neolog 6y ago
  PDF is more stable over time.

johndoe42377 6y ago

Statistics (merely observation and counting) cannot establish or even discover causation in principle. Period. No matter how many per reviewed gibberish papers will be published, the philosophical principle will stand. What you observe are effects. Causes are not there. Only discrete, fully observable, simple systems, like dice or a deck of cards, could be modelled adequately. Most of real world complex systems with multiple causation cannot.