3 ms·

This is very interesting. A discussion of the process of estimating and predicting future events, specifically going into their focus on "base rates". Apparentl

This is very interesting. A discussion of the process of estimating and predicting future events, specifically going into their focus on "base rates". Apparently they are significantly better than other groups that enter most predicting competitions. (who knew there was such a thing?) I particularly liked the idea of base rates and averaging them. [1] I can see here that the idea of them is not to be on their own accurate (some of the methodologies individually are dumb), but do get an idea for the scales of numbers to talk about (the difference between a 5% event and a 10% event is very hard to notice as a human) [1] "One was the rate at which provinces claimed by China (like Hong Kong, Macau, and Tibet) have eventually been absorbed, peacefully or by force; another was how often control of Taiwan has changed over the last few hundred years (twice; once when Japan took over from the Qing Empire in 1895 and once when the Chinese Nationalists did in 1945); the third base rate used Laplace’s rule. Laplace’s rule states that the probability of something that hasn’t happened before happening is 1 divided by N+2, where N is the number of times it hasn’t happened in the past. So the odds of the People’s Republic of China invading Taiwan this year is 1 divided by 75 (the number of years since 1949 when this has not happened) plus 2, or 1/77, or 1.3 percent. Sempere averaged his three base rates to get his initial prediction: 8 percent"

NunoSempere 3y ago
Thanks!
- danielmarkbruce 3y ago
  Out of interest - why don't you guys do way more of this for financial markets?
  - theptip 3y ago
    Presumably you don’t publish your strats there.
    - danielmarkbruce 3y ago
      I'm not sure it's quite the same thing though. It's not a strat, it's a judgement call. And, they can put a bet on at time t and then publish at time t + 1 min. This is done by hedge funds, they do publish/pitch their ideas sometimes.
  - NunoSempere 3y ago
    Yeah, good question. A few years ago the answer may have been that we thought we could influence more money than we could make. Fwiw, when I've tried it, it has tended to suck in a whole lot of my time. For me, other projects seem more meaningful. E.g., right now I'm setting up this: https://alert-team.org https://alert-team.org, another member is behind https://theaidigest.org/progress-and-dangers https://theaidigest.org/progress-and-dangers, a third works at https://forecastingresearch.org/ https://forecastingresearch.org/, etc. Maybe we're an idealistic bunch, but also maybe we're making a bad judgment call, and still yet it's possible we couldn't cut it.
    - 4ggr0 3y ago
      > Maybe we're an idealistic bunch I mean, it makes me quite happy that y'all apparently use your skills for wholesome, humanitarian things instead of just focusing on making $$$.
    - danielmarkbruce 3y ago
      Aren't you just itching to see if you can do it against the hottest competition on the planet? Ie, and I mean this with respect - financial markets are the major leagues of prediction - it feels a little bit like you are playing in the minor leagues. And it's not just a chest beating exercise - you guys appear genuine and to want to push forward the science/art of prediction. But if you can't beat the current best, you aren't state of the art, let alone able to push things forward. You might be pushing things backwards. fwiw, it appears you'd have a decent chance. You guys seem to have an extreme level of objectivity and rationality. There is a lot of domain knowledge required to bet on say, healthcare stocks or whatever, but maybe too much domain knowledge harms rather than hurts.
      - rokkitmensch 3y ago
        this Naively, if any of the predictions could be made +EV accounting for time preference, the boffins' predictions should have been mirrored into the market to someone's profit. I remain unconvinced that they're not merely very lucky, the same way that some hedge fund operators or day traders are lucky.
- hackerlight 3y ago
  Has your group made predictions on nuclear fusion timelines, or mean temperature rise due to climate change?
  - surgicalcolor 3y ago
    Eh, the second part to me seems like a less useful forecasting. Generally it seems fore asting is focused on human/political events that obviously don't already have useful modelling. For climate change, there's already a lot of existing models out there calculated by much more rigorous metrics than "the Laplace rule" or just naively adding together averages.
    - hackerlight 3y ago
      Climate models just give us scenarios. Conceptually they give us the function f(emissions) = warming + error. But we don't know the probability distribution of emissions, so we don't know the probability distribution of f. Coming up with a reliable probability distribution requires considering things like politics, world events, economic growth, technology breakthoughs, etc. IPCC etc make attempts to model the distribution of future emissions, but it's less rigorous and scientific than the actual underlying climate model.
      - exe34 3y ago
        > the probability distribution of emissions We know that every species consumes all available resources until an external force, whether it's predators or environmental collapse checks its growth. Humans are no different - any time a poor person uses a paper straw, a rich one will fly their private jet. Muh backyard arguments on nuclear means we will continue to burn coal and gas as fast as possible, even while renewables help drive up more demand. So I think it's safe to assume that emissions will continue growing.
        hackerlight 3y ago
        I'm hoping for a rigorous analysis. For example, X := "arrival time of fusion with a cost lower than natural gas in $ per kWh" is a random variable with a distribution. Our assumptions about the distribution of X will impact our estimated distribution of emissions, and hence impact our estimated distribution of f(emissions). There's lots of these questions that contribute, both technological and behavioral in nature.
lz400 3y ago
Honestly, that sounds completely arbitrary, especially the one about 1/75. It also ignores anything that would alter any thinking person's probability calculations in the real world. For example, if tomorrow the Chinese premier says that Taiwan belongs to China and starts amassing navies, the % wouldn't change. I'd personally much _much_ rather listen to a view from some scholar or expert in the geopolitical situation than these "base rate" calculations.
- valval 3y ago
  It's supposed to be a completely uninformed prior. The only information, as you can see, is introduced in the formula with the above explanation. Forecasting in general is a silly affair. As a data scientist I get all kinds of requests for all kinds of predictions, and my only real job is to part those requesting parties from their money.
- yorwba 3y ago
  The Chinese government has been saying that Taiwan belongs to China and amassed navies every year for almost 80 years now. (Longer if you count previous governments.) Base rates are important because what you think is a highly significant trigger event that will surely have major consequences may actually be a frequent occurrence that is typically inconsequential. Or in other words, if you want to say "this time is different" and deviate from the base rate, you actually need to look at the previous times to find out whether it really is different this time, and if so, by how much.
  - lz400 3y ago
    Ok, you can adjust what geopolitical event we think it's significant. Again, since I'm not an expert in the matter, I'm likely to get it wrong, as it seems I did here :) But my point stands, I'd trust expert opinion over these simplistic, falsely confident quantitative approaches.
- picadores 3y ago
  The percentages of democratic citizens assimilated into china would reach quiet a interesting percentage with taiwan assmiliated into the mainland. You can censor and propagandize all you want, but people who travelled or know better, just wont blieve a word of it happily ever after. One wonders what the internal "knowing" citizen rate is by now in china. The party relies on divide and conquer of uniformed fools. But the fools are in the minority by now - and the divide and conquer is one sneaky trustfall decentralized communication platform away from never working again.

NunoSempere 3y ago

Thanks!

danielmarkbruce 3y ago
Out of interest - why don't you guys do way more of this for financial markets?
- theptip 3y ago
  Presumably you don’t publish your strats there.
  - danielmarkbruce 3y ago
    I'm not sure it's quite the same thing though. It's not a strat, it's a judgement call. And, they can put a bet on at time t and then publish at time t + 1 min. This is done by hedge funds, they do publish/pitch their ideas sometimes.
- NunoSempere 3y ago
  Yeah, good question. A few years ago the answer may have been that we thought we could influence more money than we could make. Fwiw, when I've tried it, it has tended to suck in a whole lot of my time. For me, other projects seem more meaningful. E.g., right now I'm setting up this: https://alert-team.org https://alert-team.org, another member is behind https://theaidigest.org/progress-and-dangers https://theaidigest.org/progress-and-dangers, a third works at https://forecastingresearch.org/ https://forecastingresearch.org/, etc. Maybe we're an idealistic bunch, but also maybe we're making a bad judgment call, and still yet it's possible we couldn't cut it.
  - 4ggr0 3y ago
    > Maybe we're an idealistic bunch I mean, it makes me quite happy that y'all apparently use your skills for wholesome, humanitarian things instead of just focusing on making $$$.
  - danielmarkbruce 3y ago
    Aren't you just itching to see if you can do it against the hottest competition on the planet? Ie, and I mean this with respect - financial markets are the major leagues of prediction - it feels a little bit like you are playing in the minor leagues. And it's not just a chest beating exercise - you guys appear genuine and to want to push forward the science/art of prediction. But if you can't beat the current best, you aren't state of the art, let alone able to push things forward. You might be pushing things backwards. fwiw, it appears you'd have a decent chance. You guys seem to have an extreme level of objectivity and rationality. There is a lot of domain knowledge required to bet on say, healthcare stocks or whatever, but maybe too much domain knowledge harms rather than hurts.
    - rokkitmensch 3y ago
      this Naively, if any of the predictions could be made +EV accounting for time preference, the boffins' predictions should have been mirrored into the market to someone's profit. I remain unconvinced that they're not merely very lucky, the same way that some hedge fund operators or day traders are lucky.
hackerlight 3y ago
Has your group made predictions on nuclear fusion timelines, or mean temperature rise due to climate change?
- surgicalcolor 3y ago
  Eh, the second part to me seems like a less useful forecasting. Generally it seems fore asting is focused on human/political events that obviously don't already have useful modelling. For climate change, there's already a lot of existing models out there calculated by much more rigorous metrics than "the Laplace rule" or just naively adding together averages.
  - hackerlight 3y ago
    Climate models just give us scenarios. Conceptually they give us the function f(emissions) = warming + error. But we don't know the probability distribution of emissions, so we don't know the probability distribution of f. Coming up with a reliable probability distribution requires considering things like politics, world events, economic growth, technology breakthoughs, etc. IPCC etc make attempts to model the distribution of future emissions, but it's less rigorous and scientific than the actual underlying climate model.
    - exe34 3y ago
      > the probability distribution of emissions We know that every species consumes all available resources until an external force, whether it's predators or environmental collapse checks its growth. Humans are no different - any time a poor person uses a paper straw, a rich one will fly their private jet. Muh backyard arguments on nuclear means we will continue to burn coal and gas as fast as possible, even while renewables help drive up more demand. So I think it's safe to assume that emissions will continue growing.
      - hackerlight 3y ago
        I'm hoping for a rigorous analysis. For example, X := "arrival time of fusion with a cost lower than natural gas in $ per kWh" is a random variable with a distribution. Our assumptions about the distribution of X will impact our estimated distribution of emissions, and hence impact our estimated distribution of f(emissions). There's lots of these questions that contribute, both technological and behavioral in nature.

lz400 3y ago

Honestly, that sounds completely arbitrary, especially the one about 1/75. It also ignores anything that would alter any thinking person's probability calculations in the real world. For example, if tomorrow the Chinese premier says that Taiwan belongs to China and starts amassing navies, the % wouldn't change. I'd personally much _much_ rather listen to a view from some scholar or expert in the geopolitical situation than these "base rate" calculations.

valval 3y ago
It's supposed to be a completely uninformed prior. The only information, as you can see, is introduced in the formula with the above explanation. Forecasting in general is a silly affair. As a data scientist I get all kinds of requests for all kinds of predictions, and my only real job is to part those requesting parties from their money.
yorwba 3y ago
The Chinese government has been saying that Taiwan belongs to China and amassed navies every year for almost 80 years now. (Longer if you count previous governments.) Base rates are important because what you think is a highly significant trigger event that will surely have major consequences may actually be a frequent occurrence that is typically inconsequential. Or in other words, if you want to say "this time is different" and deviate from the base rate, you actually need to look at the previous times to find out whether it really is different this time, and if so, by how much.
- lz400 3y ago
  Ok, you can adjust what geopolitical event we think it's significant. Again, since I'm not an expert in the matter, I'm likely to get it wrong, as it seems I did here :) But my point stands, I'd trust expert opinion over these simplistic, falsely confident quantitative approaches.
picadores 3y ago
The percentages of democratic citizens assimilated into china would reach quiet a interesting percentage with taiwan assmiliated into the mainland. You can censor and propagandize all you want, but people who travelled or know better, just wont blieve a word of it happily ever after. One wonders what the internal "knowing" citizen rate is by now in china. The party relies on divide and conquer of uniformed fools. But the fools are in the minority by now - and the divide and conquer is one sneaky trustfall decentralized communication platform away from never working again.