4 ms·

Great post. Couldn't have said it any better myself. Point 3 is essential, IMHO. Superforecasting by Phillip Tetlock & Dan Gardner [1] relates an excellent desc

Great post. Couldn't have said it any better myself. Point 3 is essential, IMHO. Superforecasting by Phillip Tetlock & Dan Gardner [1] relates an excellent description of this process in the realm of human forecasting even though they don't phrase it as a Bayesian approach. Essentially they found that those best able to predict world events continuously honed their estimates using an iterative process updating what really could be described as the priors of the superforcasters. It's an enlightening read as they describe some of the processes used to hone intuited estimates using an outward and inward looking processes. I'm going to have to look into what you mean by using intuition to judge independence. Any good sources on that? [1]: https://en.m.wikipedia.org/wiki/Superforecasting https://en.m.wikipedia.org/wiki/Superforecasting

graycat 11y ago
> I'm going to have to look into what you mean by using intuition to judge independence. Any good sources on that? If look at the relevant theorems, then will conclude that intuitively random variables X and Y are independent if and only if knowledge of one of them does nothing to tell you more about the other one. If don't have independence, then can often have conditional independence, e.g., random variable X, Y are conditionally independent given Z implies E[XY|Z] = E[X|Z] E[Y|Z] So, X = number of auto accidents today in LA Y = number of auto accidents today in NY. ---------- In a family with two children X = gender of the first child Y = gender of the second child --------- At a Web site X = number of new user sessions from 10:00 AM to 10:01 AM Y = number of new user sessions from 10:01 AM to 10:02 AM ----------- On a commercial airline flight X = height of the first passenger to buy a ticket Y = height of the 10th passenger to buy a ticket Etc. Just intuition from knowledge of one of these two tells you nothing more about the other one (we assume we know the two distributions -- that we get insight into a distribution doesn't count).\ Then using independence and very little more, can use the main limit theorems, law of large numbers, central limit theorem, etc. If don't have independence, then sometimes can fake it: Given a list of n phone numbers, if toss them into a bucket and stir it and pull out the numbers and label them as X_1, X_2, ..., then in common situations can assume that the X's are independent. There are tests for independence. Likely the first to know about is a chi-squared test on a table of values of the two variables.

graycat 11y ago

> I'm going to have to look into what you mean by using intuition to judge independence. Any good sources on that? If look at the relevant theorems, then will conclude that intuitively random variables X and Y are independent if and only if knowledge of one of them does nothing to tell you more about the other one. If don't have independence, then can often have conditional independence, e.g., random variable X, Y are conditionally independent given Z implies E[XY|Z] = E[X|Z] E[Y|Z] So, X = number of auto accidents today in LA Y = number of auto accidents today in NY. ---------- In a family with two children X = gender of the first child Y = gender of the second child --------- At a Web site X = number of new user sessions from 10:00 AM to 10:01 AM Y = number of new user sessions from 10:01 AM to 10:02 AM ----------- On a commercial airline flight X = height of the first passenger to buy a ticket Y = height of the 10th passenger to buy a ticket Etc. Just intuition from knowledge of one of these two tells you nothing more about the other one (we assume we know the two distributions -- that we get insight into a distribution doesn't count).\ Then using independence and very little more, can use the main limit theorems, law of large numbers, central limit theorem, etc. If don't have independence, then sometimes can fake it: Given a list of n phone numbers, if toss them into a bucket and stir it and pull out the numbers and label them as X_1, X_2, ..., then in common situations can assume that the X's are independent. There are tests for independence. Likely the first to know about is a chi-squared test on a table of values of the two variables.