3 ms·

you get one person to guess the result of 1000 coin tosses that's a 1/(2^1000) probability of one person being right. Get a million people to guess it it's (10^

you get one person to guess the result of 1000 coin tosses that's a 1/(2^1000) probability of one person being right. Get a million people to guess it it's (10^6)/(2^1000) probability of one person being right. Lets put it in perspective; the chance of one person out of a million getting a 100% streak is 9.332 × 10^-296 %. That's significantly less than 0.000000000000000000000000001%. If there are people who have a 100% record of being correct on every guess, the probability of them being geniuses is way higher than the probability of them being lucky. That being said, I get your point. We don't have enough metrics from the research to determine whether or not these super forecasters are just lucky or geniuses.

grayclhn 11y ago
It turns out not to matter (since the numbers are so small) but your math implies that 10^305 people guessing would give a probability of about 9,300 that at least one of them is right on all 1000 coin flips. Since probabilities can't be larger than 1, that's a bit of a problem. For anyone interested, the full set of steps (that produces a numerically identical result): Prob[1 or more in 1,000,000 right] = 1 - Prob[all 1,000,000 wrong] = 1 - Prob[person 1 is wrong AND person 2 wrong AND ... person 1,000,000 wrong] = 1 - Prob[person 1 is wrong]^1,000,000 = 1 - (1 - 0.5^1000)^1,000,000 = 1 - exp(1,000,000 * log(1 - 0.5^1000)) = 1 - exp(1,000,000 * log1p(-0.5^1000)) ≈ 1 - exp(1,000,000 * -9.33 × 10^-302) = 1 - exp(-9.33 × 10^-296) = -expm1(-9.33 × 10^-296) = 9.33 × 10^-296 log1p(x) = log(1 + x) but is more accurate when x is near zero. expm1(x) = exp(x) - 1 but again is more accurate when x is near zero. Both are necessary here to get a result other than "0".
- SamReidHughes 11y ago
  It's perfectly reasonable to approximate a+b=a+b(1-a) when combining rare events without announcing it to everybody. Likewise with n repetitions of that.
  - grayclhn 11y ago
    Fair enough, but it's also reasonable to approximate 9.33 × 10^-296 as 0. I felt like doing the calculations :)
- deleted 11y ago
  [deleted]
adam 11y ago
You should read Tetlock's other work about foxes and hedgehogs in Expert Political Judgment. It presents a theory of why some people are indeed better forecasters than others. It's not lucky or genius, it's more about people who are able to consider several different data points, consider historical precedent, and make a relatively unbiased judgment about what's going to happen. These are the same types of people he sought out for the Good Judgment project, and its these same types of people who were considered Superforecasters making hundreds of forecasts over the course of a year about geopolitical issues.
aynsof 11y ago
You're right - that was a terrible example. Thanks for doing the math to prove me wrong!

grayclhn 11y ago

It turns out not to matter (since the numbers are so small) but your math implies that 10^305 people guessing would give a probability of about 9,300 that at least one of them is right on all 1000 coin flips. Since probabilities can't be larger than 1, that's a bit of a problem. For anyone interested, the full set of steps (that produces a numerically identical result): Prob[1 or more in 1,000,000 right] = 1 - Prob[all 1,000,000 wrong] = 1 - Prob[person 1 is wrong AND person 2 wrong AND ... person 1,000,000 wrong] = 1 - Prob[person 1 is wrong]^1,000,000 = 1 - (1 - 0.5^1000)^1,000,000 = 1 - exp(1,000,000 * log(1 - 0.5^1000)) = 1 - exp(1,000,000 * log1p(-0.5^1000)) ≈ 1 - exp(1,000,000 * -9.33 × 10^-302) = 1 - exp(-9.33 × 10^-296) = -expm1(-9.33 × 10^-296) = 9.33 × 10^-296 log1p(x) = log(1 + x) but is more accurate when x is near zero. expm1(x) = exp(x) - 1 but again is more accurate when x is near zero. Both are necessary here to get a result other than "0".

SamReidHughes 11y ago
It's perfectly reasonable to approximate a+b=a+b(1-a) when combining rare events without announcing it to everybody. Likewise with n repetitions of that.
- grayclhn 11y ago
  Fair enough, but it's also reasonable to approximate 9.33 × 10^-296 as 0. I felt like doing the calculations :)
deleted 11y ago
[deleted]

adam 11y ago

You should read Tetlock's other work about foxes and hedgehogs in Expert Political Judgment. It presents a theory of why some people are indeed better forecasters than others. It's not lucky or genius, it's more about people who are able to consider several different data points, consider historical precedent, and make a relatively unbiased judgment about what's going to happen. These are the same types of people he sought out for the Good Judgment project, and its these same types of people who were considered Superforecasters making hundreds of forecasts over the course of a year about geopolitical issues.

aynsof 11y ago

You're right - that was a terrible example. Thanks for doing the math to prove me wrong!