4 ms·

Frequentists vs Bayesians

30 points by thomaspaine 18y ago

jerf 18y ago
Reality is frequentist. You are small, consequently you can't do better than being a Bayesian. And you'll still be wrong. Bummer.
- joeyo 18y ago
  Maybe reality is frequentist and maybe it is not, but I will never know. After all, there can be no observer but me. Just me and my prior beliefs.
- JulianMorrison 18y ago
  If reality is deterministic, it can't possibly be frequentist. If you had total information you would assign every real event probability 1, or some reliable 0<p<1 for continuity-of-experience with a self traveling down one of the branches of an as yet unresolved quantum event.
  - jerf 18y ago
    Reality does not appear to be deterministic in any sense of the term.
    - JulianMorrison 18y ago
      Please clarify.
      - jerf 18y ago
        A vague hand wave in the direction of QM? I'm not sure how to clarify. The idea that reality is deterministic is so completely unsupported by evidence, I don't even know where to start. Between uncertainty principles forbidding you from knowing the precise state of the universe even in principle and the randomness of the universe's behavior even in situations you have full knowledge of (which may yet be traced back to mathematical undecidability, per a recent, interesting paper on arxiv), determinism is dead. If they can track it back to mathematical undecidability, no conceivable heroic effort can bring it back, either. (I suspect there's some fruitful work to be done there.)
        JulianMorrison 18y ago
        Quantum physics is usefully sketched out for the layperson here: http://www.overcomingbias.com/2008/06/the-quantum-phy.html http://www.overcomingbias.com/2008/06/the-quantum-phy.html It's deterministic. QM objections are based on bad pop sci. Undecidability in the context of physics sounds interesting, care to add detail?
        jerf 18y ago
        That series of posts goes beyond the science. I think his arguments boil down to him simply being unable to accept nondeterminism, and his argument that many-worlds is inevitable is simply absurd on the face of it; we could be in a simulation that is based in a universe without many-worlds and you'd never be able to tell the difference, so it's hardly obvious that even if we're not in a simulation that many worlds is inevitable. I also think he falls prey to Euclidianism, which most people do. In a Minkowskian universe, a lot of QM's nondeterminism is a lot less magical, even if it's still nondeterministic.
      - Retric 18y ago
        Read up on the double slit experiment in QM. http://en.wikipedia.org/wiki/Double-slit_experiment http://en.wikipedia.org/wiki/Double-slit_experiment "it is easy to account for the interference pattern that results by assuming that electrons that travel in pairs are interfering with each other because they arrive at the screen at the same time, but when a laboratory apparatus was developed that could reliably fire single electrons at the screen, the emergence of an interference pattern suggested that each electron was interfering with itself; and, therefore, in some sense the electron had to be going through both slits" PS: "it has been shown that large molecular structures like fullerene (C60) also produce interference patterns." which suggest at some level you could get the same result when firing planet's though a double slit experiment.
      - rms 18y ago
        I think you probably have different definitions of determinism. Or are thinking of different aspects of it. Or are talking about different models of QM.
Raphael 18y ago
Would someone explain this in layman's terms?
- tel 18y ago
  Disclaiming that I'm more of a statistics enthusiast than a real statistician, here's how I understand the schism. Bayesian (subjective) and Frequentist (objective) are two schools of thought about how statistics should operate. One of the best ways to think about the difference is to call Bayesians pragmatic and Frequentists rigorous. In that face of trying to quantify something you don't know, Bayesians take the stance that if you just say something — even if it's incorrect — and then keep adjusting it as more data comes in then you'll eventually have a valid statement. The idea is that even if your model is only so correct, at least you have one. Unfortunately, no one can actually prove that sort of thing actually works all of the time. It just seems to. Frequentists faced with this situation instead try to understand the reason why something is happening and then model it from the beginning. By considering these rigorous models and testing them against data they eventually build a resilient model for the unknown which validates. That is, unless they don't, in which case Frequentists are kind of out of luck. So when you're talking about statistics, which is all about trying to model things you don't understand, Bayesians and Frequentists get up in arms all the time because they each have something to call foolish about one another. The coolest part is that this sort of schism is being reflected in the world of physics as well where Frequentists are in the Newtonian/Einsteinian school but, bit by bit, that worldview is being shaken up by Quantum.
  - Eliezer 18y ago
    sorry, lkozma below got this right, you didn't. Bayesians claim to be more rigorous than frequentists, living or dying by the theorems of probability theory, rather than using a toolbox of ad-hoc tools. For example. And quantumness has very little to do with it.
    - brent 18y ago
      Arbitrary priors don't imply rigor <ducks>. :-)
Eliezer 18y ago
I don't think this article does a very good job of explaining the distinction. I know what a Bayesian is. I'm not sure that even frequentists know what a frequentist is.
- lliiffee 18y ago
  Roughly (and inaccurately) speaking, frequentism is about confidence intervals. Like, suppose someone hands me a biased coin. A frequentist can flip that coin 10 times, then make a bound on the true bias of the coin that holds with the promised degree of confidence. (E.g. they can make a bound that will hold on 95% of such experiments.) Bayesians can't do this. There are lots of other advantages to the Bayesian approach, but to frequentists, nothing could be worth giving up "coverage" (confidence intervals obeying their guarantees). P.S. When looking at those ten coin flips, and trying to estimate the true bias, the frequentist and bayesian will have the following argument: Bayesian: The probability that the true bias is b is __some formula___ Frequentist: Are you insane? The true bias of the coin is what it is! It's a number! Just because you don't know what that number is doesn't mean anything probabilistic is going on. Bayesian: Well it is extremely useful to be allowed to make such statements.
  - kurtosis 18y ago
    this is interesting - couldn't a bayesian get many of the same guarantees by computing the posterior distribution and choosing a range that contains X% of the mass?
    - eru 18y ago
      Frequentist as described in the grandparent post do all the analysis they can without having to assume priors.
    - lliiffee 18y ago
      In a word, no. The reason is that if the bayesian does a really crappy job of specifying their prior distribution, the posterior will be completely inaccurate. On the other hand, if the bayesian does a good job, getting intervals as you describe will work much better (e.g. be smaller) than confidence intervals.
brendano 18y ago
here's a better though longer frequentist vs bayesian article from brad efron. http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWModernScience.pdf http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWMode...

30 points by thomaspaine 18y ago

jerf 18y ago

Reality is frequentist. You are small, consequently you can't do better than being a Bayesian. And you'll still be wrong. Bummer.

joeyo 18y ago
Maybe reality is frequentist and maybe it is not, but I will never know. After all, there can be no observer but me. Just me and my prior beliefs.
JulianMorrison 18y ago
If reality is deterministic, it can't possibly be frequentist. If you had total information you would assign every real event probability 1, or some reliable 0<p<1 for continuity-of-experience with a self traveling down one of the branches of an as yet unresolved quantum event.
- jerf 18y ago
  Reality does not appear to be deterministic in any sense of the term.
  - JulianMorrison 18y ago
    Please clarify.
    - jerf 18y ago
      A vague hand wave in the direction of QM? I'm not sure how to clarify. The idea that reality is deterministic is so completely unsupported by evidence, I don't even know where to start. Between uncertainty principles forbidding you from knowing the precise state of the universe even in principle and the randomness of the universe's behavior even in situations you have full knowledge of (which may yet be traced back to mathematical undecidability, per a recent, interesting paper on arxiv), determinism is dead. If they can track it back to mathematical undecidability, no conceivable heroic effort can bring it back, either. (I suspect there's some fruitful work to be done there.)
      - JulianMorrison 18y ago
        Quantum physics is usefully sketched out for the layperson here: http://www.overcomingbias.com/2008/06/the-quantum-phy.html http://www.overcomingbias.com/2008/06/the-quantum-phy.html It's deterministic. QM objections are based on bad pop sci. Undecidability in the context of physics sounds interesting, care to add detail?
        jerf 18y ago
        That series of posts goes beyond the science. I think his arguments boil down to him simply being unable to accept nondeterminism, and his argument that many-worlds is inevitable is simply absurd on the face of it; we could be in a simulation that is based in a universe without many-worlds and you'd never be able to tell the difference, so it's hardly obvious that even if we're not in a simulation that many worlds is inevitable. I also think he falls prey to Euclidianism, which most people do. In a Minkowskian universe, a lot of QM's nondeterminism is a lot less magical, even if it's still nondeterministic.
    - Retric 18y ago
      Read up on the double slit experiment in QM. http://en.wikipedia.org/wiki/Double-slit_experiment http://en.wikipedia.org/wiki/Double-slit_experiment "it is easy to account for the interference pattern that results by assuming that electrons that travel in pairs are interfering with each other because they arrive at the screen at the same time, but when a laboratory apparatus was developed that could reliably fire single electrons at the screen, the emergence of an interference pattern suggested that each electron was interfering with itself; and, therefore, in some sense the electron had to be going through both slits" PS: "it has been shown that large molecular structures like fullerene (C60) also produce interference patterns." which suggest at some level you could get the same result when firing planet's though a double slit experiment.
    - rms 18y ago
      I think you probably have different definitions of determinism. Or are thinking of different aspects of it. Or are talking about different models of QM.

Raphael 18y ago

Would someone explain this in layman's terms?

tel 18y ago
Disclaiming that I'm more of a statistics enthusiast than a real statistician, here's how I understand the schism. Bayesian (subjective) and Frequentist (objective) are two schools of thought about how statistics should operate. One of the best ways to think about the difference is to call Bayesians pragmatic and Frequentists rigorous. In that face of trying to quantify something you don't know, Bayesians take the stance that if you just say something — even if it's incorrect — and then keep adjusting it as more data comes in then you'll eventually have a valid statement. The idea is that even if your model is only so correct, at least you have one. Unfortunately, no one can actually prove that sort of thing actually works all of the time. It just seems to. Frequentists faced with this situation instead try to understand the reason why something is happening and then model it from the beginning. By considering these rigorous models and testing them against data they eventually build a resilient model for the unknown which validates. That is, unless they don't, in which case Frequentists are kind of out of luck. So when you're talking about statistics, which is all about trying to model things you don't understand, Bayesians and Frequentists get up in arms all the time because they each have something to call foolish about one another. The coolest part is that this sort of schism is being reflected in the world of physics as well where Frequentists are in the Newtonian/Einsteinian school but, bit by bit, that worldview is being shaken up by Quantum.
- Eliezer 18y ago
  sorry, lkozma below got this right, you didn't. Bayesians claim to be more rigorous than frequentists, living or dying by the theorems of probability theory, rather than using a toolbox of ad-hoc tools. For example. And quantumness has very little to do with it.
  - brent 18y ago
    Arbitrary priors don't imply rigor <ducks>. :-)

Eliezer 18y ago

I don't think this article does a very good job of explaining the distinction. I know what a Bayesian is. I'm not sure that even frequentists know what a frequentist is.

lliiffee 18y ago
Roughly (and inaccurately) speaking, frequentism is about confidence intervals. Like, suppose someone hands me a biased coin. A frequentist can flip that coin 10 times, then make a bound on the true bias of the coin that holds with the promised degree of confidence. (E.g. they can make a bound that will hold on 95% of such experiments.) Bayesians can't do this. There are lots of other advantages to the Bayesian approach, but to frequentists, nothing could be worth giving up "coverage" (confidence intervals obeying their guarantees). P.S. When looking at those ten coin flips, and trying to estimate the true bias, the frequentist and bayesian will have the following argument: Bayesian: The probability that the true bias is b is __some formula___ Frequentist: Are you insane? The true bias of the coin is what it is! It's a number! Just because you don't know what that number is doesn't mean anything probabilistic is going on. Bayesian: Well it is extremely useful to be allowed to make such statements.
- kurtosis 18y ago
  this is interesting - couldn't a bayesian get many of the same guarantees by computing the posterior distribution and choosing a range that contains X% of the mass?
  - eru 18y ago
    Frequentist as described in the grandparent post do all the analysis they can without having to assume priors.
  - lliiffee 18y ago
    In a word, no. The reason is that if the bayesian does a really crappy job of specifying their prior distribution, the posterior will be completely inaccurate. On the other hand, if the bayesian does a good job, getting intervals as you describe will work much better (e.g. be smaller) than confidence intervals.

brendano 18y ago

here's a better though longer frequentist vs bayesian article from brad efron. http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWModernScience.pdf http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWMode...