36 ms·

Classical Statistics Has Outlived Its Usefulness: Here’s the Fix

9 points by johndcook 6y ago

teruakohatu 6y ago
Sigh. I find articles like this, which are written in a style so the average person can (mis)understand them, but the content requires a lot of technical knowledge. > You do not have a probability of being struck by lightning. Nobody does. I would say the average person does in fact intuitively understand that "the probability of being struck by lightning" refers to the chance of a random person being struck in a given time period (if given) or a lifetime, not themselves specifically (and definitely not if they walk around flying kites in open fields during thunderstorms). If I told someone the chance of them lifting a rock and finding a gold coin is 1*10^(-x)%, they would not lift the rock and proclaim that the coin failed to materialize, they would understand that the actual chance was zero. The patient at the start of the article not asking what are the chances this will work for me, they are asking "for men aged 18-25" what percentage of them are successfully treated with this product.
derbOac 6y ago
I admit I'm confused about what the author is arguing. They seem to be arguing that we should just adopt a strict causal predictive approach to data? I'm not sure what that really means though. They seem to be just assuming that existing observations are the population? They seem to be missing that every single estimation is a form of prediction. For example, if you estimate a regression slope parameter, you're making a prediction about the degree and nature of association of two variables in future samples. The parameter is a random variable over possible observation samples. That parameter estimate, together with the associated model (e.g., bivariate normal observations) can be translated into a specific bivariate probability distribution prediction. Lots of interesting issues are discussed but I'm not really sure they understand what they're advocating for. Statistics is always about prediction at some level, or can be formulated that way. Bayesians believe that data-based updates to models (which are used to predict) should be influenced by information available prior to the data; frequentists do not.
- throwaway81523 6y ago
  > I admit I'm confused about what the author is arguing. I suspect the author is also confused, based on the length and repetitiveness of the article. Let's say you flip a coin, have it land on the back of one of your hands, and quickly cover it with your other hand so you can't see how it landed. Despite that, it is no longer spinning: its state is now completely determined. So what is the probability P that the coin came up heads? In one way to look at it, P=50%, assuming it was a fair coin that generated a uniformly distributed binary variable. In another way, P is either 100% (the coin in fact came up heads) or else it is 0% (coin came up tails), and "50%" is meaningless: it's not Schroedinger's coin. So in this view, people who say 50% are committing the error of (insert long essay here full of fancy words like "epistomology"). Philosophers write books about this issue, plato.stanford.edu has some good articles about probability interpretation that also discuss the topic, and the linked article is more of the same. For the rest of us, there are some basic confusions that beginners and non-mathematically-informed people make, but if you take a few classes or work through a basic textbook, you can understand how significance testing works, what confidence intervals actually measure, etc, and then you can think pretty clearly. The Bayesian vs frequentist thing is a yawner: if you put one of each in a room, give them the same data and ask them to crunch the numbers, they might disagree on some philosophical abstractions but they'll get the same answers. I find Andrew Gelman's blog far more interesting than the linked one, at least based on the linked article. Either way, it's definitely worth studying some basic probability theory (understanding the math rather than trying to interpret the topic by navel-gazing) if you're a programmer or do anything else in science or technology.
wrp 6y ago
This article is based on his book Uncertainty (2016), which is a collection of various arguments about probability and statistics. Some of his arguments in the book are fairly well presented and some of them I agree with (not perfectly overlapping sets). His view of probability is sort of a combination of objective Bayesianism and logical probability. He argues that all probability is conditional on some evidence, so it is meaningless to talk of the probability of a proposition in isolation from others. Further, since probability analysis is based on relations between propositions, it is actually a part of logic. He belongs to the group who want to see hypothesis testing banished from statistics. The vast literature on that issue is easy to find. The big change he wants to see in statistics is to follow a more machine learning approach. Develop models against a sample set, then test them against a testing set rather than against some ideal distribution.

9 points by johndcook 6y ago

Sigh. I find articles like this, which are written in a style so the average person can (mis)understand them, but the content requires a lot of technical knowledge. > You do not have a probability of being struck by lightning. Nobody does. I would say the average person does in fact intuitively understand that "the probability of being struck by lightning" refers to the chance of a random person being struck in a given time period (if given) or a lifetime, not themselves specifically (and definitely not if they walk around flying kites in open fields during thunderstorms). If I told someone the chance of them lifting a rock and finding a gold coin is 1*10^(-x)%, they would not lift the rock and proclaim that the coin failed to materialize, they would understand that the actual chance was zero. The patient at the start of the article not asking what are the chances this will work for me, they are asking "for men aged 18-25" what percentage of them are successfully treated with this product.

derbOac 6y ago

I admit I'm confused about what the author is arguing. They seem to be arguing that we should just adopt a strict causal predictive approach to data? I'm not sure what that really means though. They seem to be just assuming that existing observations are the population? They seem to be missing that every single estimation is a form of prediction. For example, if you estimate a regression slope parameter, you're making a prediction about the degree and nature of association of two variables in future samples. The parameter is a random variable over possible observation samples. That parameter estimate, together with the associated model (e.g., bivariate normal observations) can be translated into a specific bivariate probability distribution prediction. Lots of interesting issues are discussed but I'm not really sure they understand what they're advocating for. Statistics is always about prediction at some level, or can be formulated that way. Bayesians believe that data-based updates to models (which are used to predict) should be influenced by information available prior to the data; frequentists do not.

throwaway81523 6y ago
> I admit I'm confused about what the author is arguing. I suspect the author is also confused, based on the length and repetitiveness of the article. Let's say you flip a coin, have it land on the back of one of your hands, and quickly cover it with your other hand so you can't see how it landed. Despite that, it is no longer spinning: its state is now completely determined. So what is the probability P that the coin came up heads? In one way to look at it, P=50%, assuming it was a fair coin that generated a uniformly distributed binary variable. In another way, P is either 100% (the coin in fact came up heads) or else it is 0% (coin came up tails), and "50%" is meaningless: it's not Schroedinger's coin. So in this view, people who say 50% are committing the error of (insert long essay here full of fancy words like "epistomology"). Philosophers write books about this issue, plato.stanford.edu has some good articles about probability interpretation that also discuss the topic, and the linked article is more of the same. For the rest of us, there are some basic confusions that beginners and non-mathematically-informed people make, but if you take a few classes or work through a basic textbook, you can understand how significance testing works, what confidence intervals actually measure, etc, and then you can think pretty clearly. The Bayesian vs frequentist thing is a yawner: if you put one of each in a room, give them the same data and ask them to crunch the numbers, they might disagree on some philosophical abstractions but they'll get the same answers. I find Andrew Gelman's blog far more interesting than the linked one, at least based on the linked article. Either way, it's definitely worth studying some basic probability theory (understanding the math rather than trying to interpret the topic by navel-gazing) if you're a programmer or do anything else in science or technology.

wrp 6y ago

This article is based on his book Uncertainty (2016), which is a collection of various arguments about probability and statistics. Some of his arguments in the book are fairly well presented and some of them I agree with (not perfectly overlapping sets). His view of probability is sort of a combination of objective Bayesianism and logical probability. He argues that all probability is conditional on some evidence, so it is meaningless to talk of the probability of a proposition in isolation from others. Further, since probability analysis is based on relations between propositions, it is actually a part of logic. He belongs to the group who want to see hypothesis testing banished from statistics. The vast literature on that issue is easy to find. The big change he wants to see in statistics is to follow a more machine learning approach. Develop models against a sample set, then test them against a testing set rather than against some ideal distribution.