7 ms·

Here's how I relax, avoid both frequentism and bayesianism, and just love probability: I assume that there is a non-empty set, commonly called Omega, which I r

by graycat 12y ago

Here's how I relax, avoid both frequentism and bayesianism, and just love probability: I assume that there is a non-empty set, commonly called Omega, which I regard as the set of all experimental 'trials' that I might observe. But, actually, in all the history of everything in the universe, we see only one trial, only one element of this set Omega. Next, there is a non-empty collection, usually denoted by script F, of subsets of Omega. I assume that set script F is contains Omega as an element and is closed under relative complements and countable unions. By relative complements, suppose A is an element of script F. Then the relative complement of set A, maybe written A^c, is essentially set Omega - A, that is, the set of all trials in Omega and not in A. Then set script F is a sigma-algebra. Each set A in script F is an event. If our trial is in set A, then we say that event A has occurred. Next there is a function P: script F --> [0, 1]. P assigns 0 to the empty set (event) and is countably additive. Then function P is a probability measure. So for each event A in script F, P(A) is a number in [0, 1] and is the probability of event A. Now can define what is means for two events to be independent and can generalize to two sigma algebras being independent. Next, on the set R of real numbers, I consider the usual topology, that is, the collection T of open subsets of R. Then I let set B, the Borel sets, be the smallest sigma algebra such that T is a subset of B. Next I consider a function X: Omega --> R such that for each Borel set A, X^{-1}(A) is an element of script F. Then X is a random variable. Essentially anything that can have a numerical value we can regard as a random variable. Then we can state and prove the classic limit theorems -- central limit theorem, weak and strong laws of large numbers, martingale convergence theorem, law of the iterated logarithm, etc. Now we are ready to do applied probability and statistics. And we have never mentioned either frequentism or Bayesianism. For more details, in an elegant presentation, see J. Neveu, Mathematical Foundations of the Calculus of Probability.

judk 12y ago
OK, now try to make some claims about the real world. You will have to make frequentist confidence or Bayesian credibility claims.
- bayesianhorse 12y ago
  Interpreting frequentist confidence in terms of the real world is actually quite difficult.
- graycat 12y ago
  No, commonly the key to "some claims about the real world" is independence. Then can apply, with meager assumptions, say, the weak law of large numbers (which we do prove), that is, take an average. Also sometimes can make some "claims' based on conditional independence (which my axioms give us the ability to define) and apply, say, the martingale convergence theorem.
throwaway000002 12y ago
Sure there's this thing called mathematics, but what reason do you have to give that the universe comes equipped with F? I'm not trying to be difficult, but mathematicians often try and live in their happy axiomatic universe and forget the axioms come from somewhere. Naïve mathematicians go further and forget that the axioms are aesthetic objects of mathematical culture and can and do change. I'm not a statistician, but like physics is to mathematics, statistics is to probability theory, and questioning the nature of the model as it corresponds to reality is part of the discipline. You can prove all the consequences you want from the axioms, but it'll never get you to closer to appreciating the relationship of those axioms and their results to an extra-theoretic "reality". Take Buffon's needle (only chosen because is interesting, and continuous, as opposed to discrete). Consider this in the real world. X = needle crosses line. You haven't dropped it yet. Is X real, how so? How does it differ from before you dropped the needle to afterwards? Deterministically X is something except that you don't know it, and suddenly you know it. In so far as that is the case, what is a reasonable way to speak about knowledge of X? For the frequentist the model is reality, i.e. you construct your sigma algebra, apply geometric arguments, crunch away, and you start glowing when you see π. What happens when you're not so sure about the model? In other words, the mathematics is always fine, but the reasoning behind the use of the mathematics is what is up for debate.
- graycat 12y ago
  In case I understand your concern, here's how I address it: We like something like probability theory because in practice it works great for saying that some roulette wheel is crooked, for confidence intervals on measurements, ..., for various stochastic processes and their power spectra, e.g., for the 3 degree K background radiation. Roughly we know quite well what we want in our probability theory. From what we know we want in our probability theory, we're essentially pushed into the axioms whether we like it or not. Or, the axioms are basically what the heck we need in order to talk about probability. E.g., we want to talk about events, say, event A, so that we can consider the probability of events, say, P(A), the probability of A. And given event B, we want to be able to consider the new event A or B. So, if in our foundations we have that events A and B are subsets of the set of trials Omega, then the new event A or B is just the set union of sets A and B. So, we want to be able to consider the new event A or B so go ahead and accept the set theory because it lets us get what we want. So, basically we got pushed into the set theory. For more, if event A is 'the next flip of our fair coin comes up heads' and event B there is 'a magnitude 1 quake in SF 10 days from now', then we jump at saying that events A and B have nothing to do with each other and are independent so that we know that P(A and B) = P(A)P(B). So, in practice, how do we justify independence? Sadly, in my view, mostly just intuitively as in the little example of coins and quakes I gave. The axioms ask for a little more than we might, first cut, have believed we need to specify. E.g., so that we will have plenty of ability to take old events and create new ones, we want the events actually to form a sigma algebra. Next, so that we can talk about the event 'the coin never comes up heads', again we want a sigma algebra, and we want our probability P to be countably additive. What about uncountably additive? Quickly we see that that causes us serious problems so don't ask for it. Net, we really do like probability and its applications, say, to statistics, and the axioms I gave (thank you A. Kolmogorov and H. Lebesgue) are basically about the minimum we want. Really, we just get pushed into those axioms; if we like probability, and we do, then we don't have much choice but to accept those axioms. Curious universe we live in.
grayclhn 12y ago
Of course we haven't mentioned Bayesianism or Frequentism yet... you haven't said anything about estimating aspects of the probability space from observed data.
- graycat 12y ago
  > estimating aspects Use the classic limit theorems, especially just the weak law of large numbers, i.e., take an average.
  - grayclhn 12y ago
    ...and now you have a naive version of frequentist estimation.
    - graycat 12y ago
      Well, we need an independence assumption. And then we have some rock solid theorems and not just something "naive". Maybe we have to pull the rabbit out of the hat someplace, and with the axioms our place is an independence assumption. Indeed, the heartburn when trying to swallow frequentist is that we don't have the crucial assumptions to make an average have valuable properties; the sufficient and standard assumption is independence, and then the laws of large numbers, which we can prove just as theorems from the axioms and an independence assumption, tell us that an average does give us what we want to make the whole subject go, in statistics, etc. So, net, where we pull the rabbit out of the hat is the independence assumption. If we are willing to assume independence, then we don't have to address the philosophical points, or we move the consideration of the philosophical points to considering the independence assumption. But, we are forced: We know that we want averages to be good estimators and know that in practice they often are; since the frequentist approach does not have enough to let us be clear on the properties of an average, we need more; a way to get what we need is the axioms and an independence assumption. To me the key to probability, and why it is really not the same as just Lebesgue's measure theory, is the role of independence. We can also add in conditional independence. Net, I don't think that we can get very far in probability or its foundations without something as powerful as independence (or at least uncorrelated, which independence implies). The axioms and independence make a nice package to give us a rock solid version of what we want. So, that's why I go for it.

by graycat 12y ago

judk 12y ago

OK, now try to make some claims about the real world. You will have to make frequentist confidence or Bayesian credibility claims.

bayesianhorse 12y ago
Interpreting frequentist confidence in terms of the real world is actually quite difficult.
graycat 12y ago
No, commonly the key to "some claims about the real world" is independence. Then can apply, with meager assumptions, say, the weak law of large numbers (which we do prove), that is, take an average. Also sometimes can make some "claims' based on conditional independence (which my axioms give us the ability to define) and apply, say, the martingale convergence theorem.

throwaway000002 12y ago

Sure there's this thing called mathematics, but what reason do you have to give that the universe comes equipped with F? I'm not trying to be difficult, but mathematicians often try and live in their happy axiomatic universe and forget the axioms come from somewhere. Naïve mathematicians go further and forget that the axioms are aesthetic objects of mathematical culture and can and do change. I'm not a statistician, but like physics is to mathematics, statistics is to probability theory, and questioning the nature of the model as it corresponds to reality is part of the discipline. You can prove all the consequences you want from the axioms, but it'll never get you to closer to appreciating the relationship of those axioms and their results to an extra-theoretic "reality". Take Buffon's needle (only chosen because is interesting, and continuous, as opposed to discrete). Consider this in the real world. X = needle crosses line. You haven't dropped it yet. Is X real, how so? How does it differ from before you dropped the needle to afterwards? Deterministically X is something except that you don't know it, and suddenly you know it. In so far as that is the case, what is a reasonable way to speak about knowledge of X? For the frequentist the model is reality, i.e. you construct your sigma algebra, apply geometric arguments, crunch away, and you start glowing when you see π. What happens when you're not so sure about the model? In other words, the mathematics is always fine, but the reasoning behind the use of the mathematics is what is up for debate.

graycat 12y ago
In case I understand your concern, here's how I address it: We like something like probability theory because in practice it works great for saying that some roulette wheel is crooked, for confidence intervals on measurements, ..., for various stochastic processes and their power spectra, e.g., for the 3 degree K background radiation. Roughly we know quite well what we want in our probability theory. From what we know we want in our probability theory, we're essentially pushed into the axioms whether we like it or not. Or, the axioms are basically what the heck we need in order to talk about probability. E.g., we want to talk about events, say, event A, so that we can consider the probability of events, say, P(A), the probability of A. And given event B, we want to be able to consider the new event A or B. So, if in our foundations we have that events A and B are subsets of the set of trials Omega, then the new event A or B is just the set union of sets A and B. So, we want to be able to consider the new event A or B so go ahead and accept the set theory because it lets us get what we want. So, basically we got pushed into the set theory. For more, if event A is 'the next flip of our fair coin comes up heads' and event B there is 'a magnitude 1 quake in SF 10 days from now', then we jump at saying that events A and B have nothing to do with each other and are independent so that we know that P(A and B) = P(A)P(B). So, in practice, how do we justify independence? Sadly, in my view, mostly just intuitively as in the little example of coins and quakes I gave. The axioms ask for a little more than we might, first cut, have believed we need to specify. E.g., so that we will have plenty of ability to take old events and create new ones, we want the events actually to form a sigma algebra. Next, so that we can talk about the event 'the coin never comes up heads', again we want a sigma algebra, and we want our probability P to be countably additive. What about uncountably additive? Quickly we see that that causes us serious problems so don't ask for it. Net, we really do like probability and its applications, say, to statistics, and the axioms I gave (thank you A. Kolmogorov and H. Lebesgue) are basically about the minimum we want. Really, we just get pushed into those axioms; if we like probability, and we do, then we don't have much choice but to accept those axioms. Curious universe we live in.

grayclhn 12y ago

Of course we haven't mentioned Bayesianism or Frequentism yet... you haven't said anything about estimating aspects of the probability space from observed data.

graycat 12y ago
> estimating aspects Use the classic limit theorems, especially just the weak law of large numbers, i.e., take an average.
- grayclhn 12y ago
  ...and now you have a naive version of frequentist estimation.
  - graycat 12y ago
    Well, we need an independence assumption. And then we have some rock solid theorems and not just something "naive". Maybe we have to pull the rabbit out of the hat someplace, and with the axioms our place is an independence assumption. Indeed, the heartburn when trying to swallow frequentist is that we don't have the crucial assumptions to make an average have valuable properties; the sufficient and standard assumption is independence, and then the laws of large numbers, which we can prove just as theorems from the axioms and an independence assumption, tell us that an average does give us what we want to make the whole subject go, in statistics, etc. So, net, where we pull the rabbit out of the hat is the independence assumption. If we are willing to assume independence, then we don't have to address the philosophical points, or we move the consideration of the philosophical points to considering the independence assumption. But, we are forced: We know that we want averages to be good estimators and know that in practice they often are; since the frequentist approach does not have enough to let us be clear on the properties of an average, we need more; a way to get what we need is the axioms and an independence assumption. To me the key to probability, and why it is really not the same as just Lebesgue's measure theory, is the role of independence. We can also add in conditional independence. Net, I don't think that we can get very far in probability or its foundations without something as powerful as independence (or at least uncorrelated, which independence implies). The axioms and independence make a nice package to give us a rock solid version of what we want. So, that's why I go for it.