6 ms·

It appears to me that the reason Bayesian probability is somewhat elusive is due to its metaphysical underpinning: what do we mean by probability? If we live i

It appears to me that the reason Bayesian probability is somewhat elusive is due to its metaphysical underpinning: what do we mean by probability? If we live in a materialist deterministic world - which many would cite as an axiom for simplicity - then there really is no probability. Everything happens with 100% certainty. Then, what is probability? If everything will happen with 100% certainty, but probability certainly appears to exist, then probability must reflect something about our information about something occurring. The author refers to two foundational approaches to our state of knowledge. The first is the objectivist approach, which states that everyone who has the same state of knowledge about a system will evaluate the same probability of something occurring. The second is the subjectivist approach, which states that a given individual with a certain state of knowledge will evaluate some probability of something occurring. To me, these appear to be the same thing except insofar as the former requires a consensus of many while the latter a consensus of one. The author asks how we might actually define Bayesian probability without resorting to the frequentist approach (i.e. hypothetically simulating many trials of the same event, however infrequent in reality it may be). First, he says this would mean "interpreting [the credible interval] like a confidence interval". I am no statistician, but is that necessarily true? I don't see why confidence intervals would suddenly emerge in order to interpret a credible interval. Second, I am not sure the frequentist interpretation is so problematic. When we interpret the plain-English definition of a probability, are we not mentally simulating repeated trials in order to evaluate something's occurrence? What else could a probability imply? If something has a 20% chance of occurring, then it does not occur 80% of the time, and so we must envision 80% of universes (part of the hypothetical trials) where it does not occur. I don't see any other way around this, frequentist or not. (Note: I am not a statistician, while the author is, and the above is simply my layman's understanding of the article.)

outlace 6y ago
It seems a Bayesian interpretation of probability is more general. The frequency of events over an infinite number of trials is one way of interpreting probability for things that are able to be repeated. But this wouldn’t make sense to apply for an election that is only going to happen once and yet one still wants to be able to quantify uncertainty in these situations.
- KKPMW 6y ago
  There are a few things intermingled in this election example. 1. The outcome of the election here is not a probability. It is the population value - the ratio of people voting for candidate X on the election date. It doesn't have to be repeated in the same way measurements of height for all people in United States would not have to be repeated, if instead of vote we were measuring heights. 2. Frequentist probability doesn't require to physically repeat things. It can reason about what would happen in the repeated sampling under certain conditions, and then draw inferences about those assumed conditions. With the election example: if you get a survey of 100 people with 70% voting for candidate "A" we don't need to repeat this survey in order to know the likelihood (frequency) of this result happening if the real proportion of people voting for candidate "A" across the US is 50%.
  - outlace 6y ago
    If you’re trying to quantify your uncertainty about who will win the election, a poll would only be part of it. You want to be able to combine disparate sources of information. Maybe there is preference falsification and you want to incorporate as some sort of prior. As things get further from simple sampling from a population the frequency interpretation makes less and less sense to me.
    - KKPMW 6y ago
      Sorry, I don't see how this reply is related to the original objection that frequentist methods are not applicable to events that happen once.
maire 6y ago
> If we live in a materialist deterministic world - which many would cite as an axiom for simplicity - then there really is no probability. Everything happens with 100% certainty. I don't know who "the many" are - but I thought determinism had already been disproved. I am not a physicist so I will not go into quantum mechanics - but I will take a simple example from Science Fiction, and that is the Temporal Paradox. https://en.wikipedia.org/wiki/Temporal_paradox https://en.wikipedia.org/wiki/Temporal_paradox
- mjburgess 6y ago
  How does this disprove determinism?
  - maire 6y ago
    Sorry - I was being flippant because your argument involved time travel. After the fact an event has 100% probability that it did occur but before the event it does not. The entire field of chaos theory is to make chaos deterministic so you are in good company. There is no general mechanism (yet) to do this. Quantum mechanics is the most interesting area. The measure of determinism is that it predicts the future before it happens. People have been trying to do this for years in weather and the stock market. This is where the concept of Chaos came from. Leaving math and physics - there is philosophy. To apply determinism to people you would have to decide there is no free will. Maybe this is true and maybe not.
cultus 6y ago
There's no definition of probability that doesn't involve philosophy or metaphysics [0]. Calling frequentist stats "objective" really bugs me. There is no such thing. Every inference procedure involves subjective choices. Of course, frequentist and Bayesian stats are completely mathematically equivalent. The choice just affects our mental patterns. [0] https://en.wikipedia.org/wiki/Probability_interpretations#Philosophy https://en.wikipedia.org/wiki/Probability_interpretations#Ph...
- radford-neal 6y ago
  No, frequentist and Bayesian statistics are not equivalent. There are some special cases in which a frequentist 95% confidence interval and a Bayesian 95% credible interval based on some sort of default prior are numerically the same, but that doesn't happen in general. Statisticians would hardly have been vigorously debating the issue for two centuries if it didn't really matter.
  - analog31 6y ago
    From the article: >>>> Let’s now suppose that we’ve done a Bayesian analysis. We’ve specified a prior distribution for the parameter, based on prior evidence, our subjective beliefs about the value of the parameter, or perhaps we used a default ‘non-informative’ prior built into our software package. At first blush the difference is that the Bayesian is using more information. Now don't get me wrong, if Bayes theorem and its progeny give us useful tools for incorporating that information in our analyses, so much the better.
  - 08-15 6y ago
    It's been only one century since Ronald Fisher misunderstood statistics and thereby created what's now known as 'frequentist statistics'.
  - cultus 6y ago
    Every frequentist technique has a Bayesian interpretation and vice-versa. Confidence intervals are equivalent to credible intervals with certain priors.
6gvONxR4sf7o 6y ago
You’re correct that probability still works in a hypothesized deterministic universe. It’s a point that’s often too often forgotten, causing discussions to go down unnecessary rabbit holes debating foundations of quantum mechanics when discussing the roll of a six sided die. Statisticians and mathematicians have gone very far down the path you’re discussing, and you might be interested in some sets of axioms that have come up around probability and relaxations of true/false logic. The Kolmogorov axioms [0] are the “standard” probability axioms, and are phrased in terms of set theory and measure theory (not requiring any mention of physics or a physical universe!). There are other ways to quantify degree of belief, however, and they are very interesting. Apparently Cox’s theorem [1] justifies a popular probability framework for Bayesians. But there are many more interesting ways to do degree of belief, like Dempster-Schafer theory [2], which I understand to be a plausibility calculus. Everybody seems to find a single system and decide it’s the only one out there, [0] https://en.wikipedia.org/wiki/Probability_axioms https://en.wikipedia.org/wiki/Probability_axioms [1] https://en.wikipedia.org/wiki/Cox's_theorem https://en.wikipedia.org/wiki/Cox's_theorem [2] https://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory https://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory

outlace 6y ago

It seems a Bayesian interpretation of probability is more general. The frequency of events over an infinite number of trials is one way of interpreting probability for things that are able to be repeated. But this wouldn’t make sense to apply for an election that is only going to happen once and yet one still wants to be able to quantify uncertainty in these situations.

KKPMW 6y ago
There are a few things intermingled in this election example. 1. The outcome of the election here is not a probability. It is the population value - the ratio of people voting for candidate X on the election date. It doesn't have to be repeated in the same way measurements of height for all people in United States would not have to be repeated, if instead of vote we were measuring heights. 2. Frequentist probability doesn't require to physically repeat things. It can reason about what would happen in the repeated sampling under certain conditions, and then draw inferences about those assumed conditions. With the election example: if you get a survey of 100 people with 70% voting for candidate "A" we don't need to repeat this survey in order to know the likelihood (frequency) of this result happening if the real proportion of people voting for candidate "A" across the US is 50%.
- outlace 6y ago
  If you’re trying to quantify your uncertainty about who will win the election, a poll would only be part of it. You want to be able to combine disparate sources of information. Maybe there is preference falsification and you want to incorporate as some sort of prior. As things get further from simple sampling from a population the frequency interpretation makes less and less sense to me.
  - KKPMW 6y ago
    Sorry, I don't see how this reply is related to the original objection that frequentist methods are not applicable to events that happen once.

maire 6y ago

> If we live in a materialist deterministic world - which many would cite as an axiom for simplicity - then there really is no probability. Everything happens with 100% certainty. I don't know who "the many" are - but I thought determinism had already been disproved. I am not a physicist so I will not go into quantum mechanics - but I will take a simple example from Science Fiction, and that is the Temporal Paradox. https://en.wikipedia.org/wiki/Temporal_paradox https://en.wikipedia.org/wiki/Temporal_paradox

mjburgess 6y ago
How does this disprove determinism?
- maire 6y ago
  Sorry - I was being flippant because your argument involved time travel. After the fact an event has 100% probability that it did occur but before the event it does not. The entire field of chaos theory is to make chaos deterministic so you are in good company. There is no general mechanism (yet) to do this. Quantum mechanics is the most interesting area. The measure of determinism is that it predicts the future before it happens. People have been trying to do this for years in weather and the stock market. This is where the concept of Chaos came from. Leaving math and physics - there is philosophy. To apply determinism to people you would have to decide there is no free will. Maybe this is true and maybe not.

cultus 6y ago

There's no definition of probability that doesn't involve philosophy or metaphysics [0]. Calling frequentist stats "objective" really bugs me. There is no such thing. Every inference procedure involves subjective choices. Of course, frequentist and Bayesian stats are completely mathematically equivalent. The choice just affects our mental patterns. [0] https://en.wikipedia.org/wiki/Probability_interpretations#Philosophy https://en.wikipedia.org/wiki/Probability_interpretations#Ph...

radford-neal 6y ago
No, frequentist and Bayesian statistics are not equivalent. There are some special cases in which a frequentist 95% confidence interval and a Bayesian 95% credible interval based on some sort of default prior are numerically the same, but that doesn't happen in general. Statisticians would hardly have been vigorously debating the issue for two centuries if it didn't really matter.
- analog31 6y ago
  From the article: >>>> Let’s now suppose that we’ve done a Bayesian analysis. We’ve specified a prior distribution for the parameter, based on prior evidence, our subjective beliefs about the value of the parameter, or perhaps we used a default ‘non-informative’ prior built into our software package. At first blush the difference is that the Bayesian is using more information. Now don't get me wrong, if Bayes theorem and its progeny give us useful tools for incorporating that information in our analyses, so much the better.
- 08-15 6y ago
  It's been only one century since Ronald Fisher misunderstood statistics and thereby created what's now known as 'frequentist statistics'.
- cultus 6y ago
  Every frequentist technique has a Bayesian interpretation and vice-versa. Confidence intervals are equivalent to credible intervals with certain priors.

6gvONxR4sf7o 6y ago

You’re correct that probability still works in a hypothesized deterministic universe. It’s a point that’s often too often forgotten, causing discussions to go down unnecessary rabbit holes debating foundations of quantum mechanics when discussing the roll of a six sided die. Statisticians and mathematicians have gone very far down the path you’re discussing, and you might be interested in some sets of axioms that have come up around probability and relaxations of true/false logic. The Kolmogorov axioms [0] are the “standard” probability axioms, and are phrased in terms of set theory and measure theory (not requiring any mention of physics or a physical universe!). There are other ways to quantify degree of belief, however, and they are very interesting. Apparently Cox’s theorem [1] justifies a popular probability framework for Bayesians. But there are many more interesting ways to do degree of belief, like Dempster-Schafer theory [2], which I understand to be a plausibility calculus. Everybody seems to find a single system and decide it’s the only one out there, [0] https://en.wikipedia.org/wiki/Probability_axioms https://en.wikipedia.org/wiki/Probability_axioms [1] https://en.wikipedia.org/wiki/Cox's_theorem https://en.wikipedia.org/wiki/Cox's_theorem [2] https://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory https://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory