6 ms·

> Well, we need an independence assumption. And then we have some rock solid theorems and not just something "naive". We have rock solid theorems that the ave

> Well, we need an independence assumption. And then we have some rock solid theorems and not just something "naive". We have rock solid theorems that the average converges under some conditions. (As an aside, do you rule out the Cauchy distribution in your axiomatic approach?). We don't have rock solid theorems that the average is of any interest. If we're interested in knowing whether a dam is likely to overflow based on historical water levels, the average is of limited use -- for heights well below the highest historical depth, an average will be something that we can at least calculate, but not precisely. If we want to leave room for error (build a dam above the highest historical waterlevel, and do it conservatively so that we'll be above future maixima for the next x years w/ some probability), we can't even calculate an average. Now what? That's what I mean by "naive." If we're going to try to attack problems that aren't transparently easy (something like stationary weakly-dependent data with finite moments, enough observations that not only the CLT is a good approximation, but that we don't even need to worry about efficiency, loosely defined) then we need to start thinking about the properties we want our estimation procedure to have. One set of properties leads to something like frequentist statistics, another set of properties leads to Bayesian statistics. Other sets of properties lead to some of the more idiosyncratic branches of statistics that I don't know much about. > since the frequentist approach does not have enough to let us be clear on the properties of an average, we need more This makes no sense. Every branch of statistics puts the estimator on top of a probability space, just usually not explicitly. (Because it would be boring.) The t-test has certain properties if the probability space is in a class with.... > The axioms and independence make a nice package to give us a rock solid version of what we want. So, that's why I go for it. I want to get decent answers for hard problems and have a measure of their reliability. Both frequentist and Bayesian stats are often good for this, and certainly better than "no stats."

graycat 12y ago
> (As an aside, do you rule out the Cauchy distribution in your axiomatic approach?). Of course not. Essentially the only assumption about a random variable is measurability, and I essentially gave that. Each of the limit theorems has assumptions, and in some cases minimal assumptions are difficult to consider, e.g., for the Lindeberg-Feller version of the central limit theorem. For a random variable with the Cauchy distribution, as I recall, the integral of the positive part is infinity and that of the negative part is infinity so that to define its expectation we would have to subtract one infinity from another which, with the measure theory approach to 'improper' integrals, we are not willing to do. So, that random variable does not have an expectation, and the law of large numbers does not apply to an independent, identically distributed sequence of such random variables. Again, > We don't have rock solid theorems that the average is of any interest. Sure, we do: Maybe for real valued random variables X and Y and real number a, we are told that X = a + Y and want to find a. Suppose we have E[X] exists and is finite and for i = 1, 2, ..., we have real value random variables X(i), independent and distributed like X. So, then we also have independent Y(i) distributed like Y. Suppose we are told that E[Y] = 0. Then we can use the law of large numbers to estimate a. > If we're interested in knowing whether a dam is likely to overflow based on historical water levels, the average is of limited use -- for heights well below the highest historical depth, an average will be something that we can at least calculate, but not precisely. If we have random variable X that is 1 when the dam overflows and 0 otherwise, then the probability the dam overflows is just E[X]. If we have independent, identically distributed 'samples' of X, that is, X(i) for i = 1, 2, ..., then we can use the weak law of large numbers to estimate E[X]. The role of "historical water levels" here is not trivial to evaluate; the main reason is that we are not sure we have the values of a sequence of independent, identically distributed random variables. Generally from experience in applied probability and statistics, we should know well that saying what the probability of some particular water level is in the next 100 years is a challenging problem with no royal road to a solution and not a weakness of the axioms, expectation, or the laws of large numbers. > This makes no sense. Every branch of statistics puts the estimator on top of a probability space, just usually not explicitly. (Because it would be boring.) The t-test has certain properties if the probability space is in a class with.... With the axioms I gave, independence, etc., the Student's t distribution and its applications work just fine as long practiced without mention of either frequentist or Bayesianism. The reference I gave, Neveu, uses the axiomatic approach I outlined. So do the other famous texts in 'graduate probability' by Loeve, Breiman, and Chung. So, let's see: I just got out my copy of Neveu and checked the index. 'Frequentist' is never mentioned, and Bayes is mentioned only on one page and there only as a 'strategy' in the "formalism" of statistical decision theory. Net, beyond the simple Bayes rule P(A|B) = P(A) P(B|A)/P(B) which is immediate from the definition P(A|B) = P(A and B)/P(B) really, Bayes has next to no role in the whole book. Net, with the axiomatic foundation I outlined, we don't have to struggle, consider, or even mention either frequentist of Bayesianism. That should be welcome good news.
- kgwgk 12y ago
  > Net, with the axiomatic foundation I outlined, we don't have to struggle, consider, or even mention either frequentist of Bayesianism. That should be welcome good news. People have been building on top of the axiomatic foundation that you outlined for decades. This is old news. You mention Breiman's book on Probability. He has also a book on Statistics. Maybe there is something to it after all?
  - graycat 12y ago
    > People have been building on top of the axiomatic foundation that you outlined for decades. This is old news. Yup, essentially since Kolmogorov's 1933 paper. The axiomatic approach I gave is just what is used in Breiman's Probability, as published by SIAM. It's a super nicely written book. I have never seen his book on statistics.
    - kgwgk 12y ago
      I can't resist quoting Jaynes (from the closing remarks in Chapter 2 of "Probability Theory: The Logic of Science"): In 1933, A. N. Kolmogorov presented an approach to probability theory phrased in the language of set theory and measure theory. This language was just then becoming so fashionable that today many mathematical results are named, not for the discoverer, but for the one who first restated them in that language. For example, in the theory of continuous groups the term “Hurwitz invariant integral” disappeared, to be replaced by “Haar measure.” Because of this custom, some modern works—particularly by mathematicians—can give one the impression that probability theory started with Kolmogorov. [...] However, our system of probability differs conceptually from that of Kolmogorov in that we do not interpret propositions in terms of sets, but we do interpret probability distributions as carriers of incomplete information. Partly as a result, our system has analytical resources not present at all in the Kolmogorov system. This enables us to formulate and solve many problems—particularly the so-called “ill posed” problems and “generalized inverse” problems—that would be considered outside the scope of probability theory according to the Kolmogorov system. These problems are just the ones of greatest interest in current applications.

graycat 12y ago

> (As an aside, do you rule out the Cauchy distribution in your axiomatic approach?). Of course not. Essentially the only assumption about a random variable is measurability, and I essentially gave that. Each of the limit theorems has assumptions, and in some cases minimal assumptions are difficult to consider, e.g., for the Lindeberg-Feller version of the central limit theorem. For a random variable with the Cauchy distribution, as I recall, the integral of the positive part is infinity and that of the negative part is infinity so that to define its expectation we would have to subtract one infinity from another which, with the measure theory approach to 'improper' integrals, we are not willing to do. So, that random variable does not have an expectation, and the law of large numbers does not apply to an independent, identically distributed sequence of such random variables. Again, > We don't have rock solid theorems that the average is of any interest. Sure, we do: Maybe for real valued random variables X and Y and real number a, we are told that X = a + Y and want to find a. Suppose we have E[X] exists and is finite and for i = 1, 2, ..., we have real value random variables X(i), independent and distributed like X. So, then we also have independent Y(i) distributed like Y. Suppose we are told that E[Y] = 0. Then we can use the law of large numbers to estimate a. > If we're interested in knowing whether a dam is likely to overflow based on historical water levels, the average is of limited use -- for heights well below the highest historical depth, an average will be something that we can at least calculate, but not precisely. If we have random variable X that is 1 when the dam overflows and 0 otherwise, then the probability the dam overflows is just E[X]. If we have independent, identically distributed 'samples' of X, that is, X(i) for i = 1, 2, ..., then we can use the weak law of large numbers to estimate E[X]. The role of "historical water levels" here is not trivial to evaluate; the main reason is that we are not sure we have the values of a sequence of independent, identically distributed random variables. Generally from experience in applied probability and statistics, we should know well that saying what the probability of some particular water level is in the next 100 years is a challenging problem with no royal road to a solution and not a weakness of the axioms, expectation, or the laws of large numbers. > This makes no sense. Every branch of statistics puts the estimator on top of a probability space, just usually not explicitly. (Because it would be boring.) The t-test has certain properties if the probability space is in a class with.... With the axioms I gave, independence, etc., the Student's t distribution and its applications work just fine as long practiced without mention of either frequentist or Bayesianism. The reference I gave, Neveu, uses the axiomatic approach I outlined. So do the other famous texts in 'graduate probability' by Loeve, Breiman, and Chung. So, let's see: I just got out my copy of Neveu and checked the index. 'Frequentist' is never mentioned, and Bayes is mentioned only on one page and there only as a 'strategy' in the "formalism" of statistical decision theory. Net, beyond the simple Bayes rule P(A|B) = P(A) P(B|A)/P(B) which is immediate from the definition P(A|B) = P(A and B)/P(B) really, Bayes has next to no role in the whole book. Net, with the axiomatic foundation I outlined, we don't have to struggle, consider, or even mention either frequentist of Bayesianism. That should be welcome good news.

kgwgk 12y ago
> Net, with the axiomatic foundation I outlined, we don't have to struggle, consider, or even mention either frequentist of Bayesianism. That should be welcome good news. People have been building on top of the axiomatic foundation that you outlined for decades. This is old news. You mention Breiman's book on Probability. He has also a book on Statistics. Maybe there is something to it after all?
- graycat 12y ago
  > People have been building on top of the axiomatic foundation that you outlined for decades. This is old news. Yup, essentially since Kolmogorov's 1933 paper. The axiomatic approach I gave is just what is used in Breiman's Probability, as published by SIAM. It's a super nicely written book. I have never seen his book on statistics.
  - kgwgk 12y ago
    I can't resist quoting Jaynes (from the closing remarks in Chapter 2 of "Probability Theory: The Logic of Science"): In 1933, A. N. Kolmogorov presented an approach to probability theory phrased in the language of set theory and measure theory. This language was just then becoming so fashionable that today many mathematical results are named, not for the discoverer, but for the one who first restated them in that language. For example, in the theory of continuous groups the term “Hurwitz invariant integral” disappeared, to be replaced by “Haar measure.” Because of this custom, some modern works—particularly by mathematicians—can give one the impression that probability theory started with Kolmogorov. [...] However, our system of probability differs conceptually from that of Kolmogorov in that we do not interpret propositions in terms of sets, but we do interpret probability distributions as carriers of incomplete information. Partly as a result, our system has analytical resources not present at all in the Kolmogorov system. This enables us to formulate and solve many problems—particularly the so-called “ill posed” problems and “generalized inverse” problems—that would be considered outside the scope of probability theory according to the Kolmogorov system. These problems are just the ones of greatest interest in current applications.