5 ms·

It is an unfortunate misconception that statistical probability can be used to predict the future. Any time you extend a statistical model temporally it immedi

It is an unfortunate misconception that statistical probability can be used to predict the future. Any time you extend a statistical model temporally it immediately becomes mathematically invalid since probabilistic statistics are only valid for a fixed population at a fixed moment in time. Unfortunately business and government is rife with people predicting the future based on statistical models that have no more mathematical validity than reading tea leaves.

gbrown 9y ago
What??? Prediction is certainly a type of extrapolation, but to claim that it's "mathematically invalid" reveals a severe lack of knowledge on your part. In fact, under parametric assumptions about the data generating mechanism, we can exactly quantify the expected coverage of prediction intervals. That's literally a standard topic in an introductory statistics course.
- sgt101 9y ago
  Hello, I think Calafrax is probably right. :o) I think you implicitly agree because you say "under parametric assumptions..." which means you know whats going on; but to make the point-> Statistics as we know it "works" (can be derived) under the assumptions of controlled experimental data. As a thought experiment think about the weather - we know that if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly. Why - well because weather is very very very complex; the range of behavior is vast. But worse, it's unstable. The weather in my garden is driven by several complex systems; the ocean, the atmosphere, the earth's orbit and sol! Statistics can't predict the future of the weather in my garden. Statistics also can't predict other things like the future of the financial markets (not least because if you find a statistical law about that they you will act on it and then screw it up) It's important to me to bang on about this because there are loads of people who sit through their introductory courses and read the example of predicting a biased roulette wheel. Years later they end up running the company/country/community that I live in and they have a view that they can use the same principles to do it... and this thinking leads to nasty surprises for me.
  - panarky 9y ago
    > if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly Give me hourly readings of temperature, wind speed, wind direction, precipitation, cloud cover and barometric pressure for the last 10 years and I can give you a very accurate prediction of tomorrow's weather in your garden.
    - calafrax 9y ago
      is that a joke? weather predictions are notoriously unreliable even though they are given with extreme granularity. that aside you are missing a larger point. if you predict the future based on past data all you are saying is "the future will be the same as the past." you aren't predicting anything. you will be wrong every single time something novel occurs, which is pretty frequently in the real world.
      - ryanwaggoner 9y ago
        The perception that weather forecasting is notoriously unreliable is mostly false: https://mobile.nytimes.com/2012/09/09/magazine/the-weatherman-is-not-a-moron.html?referer= https://mobile.nytimes.com/2012/09/09/magazine/the-weatherma...
        sgt101 9y ago
        From your link : "Why are weather forecasters succeeding when other predictors fail? It’s because long ago they came to accept the imperfections in their knowledge. That helped them understand that even the most sophisticated computers, combing through seemingly limitless data, are painfully ill equipped to predict something as dynamic as weather all by themselves. So as fields like economics began relying more on Big Data, meteorologists recognized that data on its own isn’t enough."
        gbrown 9y ago
        Quantifying uncertainty is one of the main points of statistics. Don't confuse the limitations of point estimates provided by machine learning techniques with all of statistical practice.

gbrown 9y ago

What??? Prediction is certainly a type of extrapolation, but to claim that it's "mathematically invalid" reveals a severe lack of knowledge on your part. In fact, under parametric assumptions about the data generating mechanism, we can exactly quantify the expected coverage of prediction intervals. That's literally a standard topic in an introductory statistics course.

sgt101 9y ago
Hello, I think Calafrax is probably right. :o) I think you implicitly agree because you say "under parametric assumptions..." which means you know whats going on; but to make the point-> Statistics as we know it "works" (can be derived) under the assumptions of controlled experimental data. As a thought experiment think about the weather - we know that if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly. Why - well because weather is very very very complex; the range of behavior is vast. But worse, it's unstable. The weather in my garden is driven by several complex systems; the ocean, the atmosphere, the earth's orbit and sol! Statistics can't predict the future of the weather in my garden. Statistics also can't predict other things like the future of the financial markets (not least because if you find a statistical law about that they you will act on it and then screw it up) It's important to me to bang on about this because there are loads of people who sit through their introductory courses and read the example of predicting a biased roulette wheel. Years later they end up running the company/country/community that I live in and they have a view that they can use the same principles to do it... and this thinking leads to nasty surprises for me.
- panarky 9y ago
  > if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly Give me hourly readings of temperature, wind speed, wind direction, precipitation, cloud cover and barometric pressure for the last 10 years and I can give you a very accurate prediction of tomorrow's weather in your garden.
  - calafrax 9y ago
    is that a joke? weather predictions are notoriously unreliable even though they are given with extreme granularity. that aside you are missing a larger point. if you predict the future based on past data all you are saying is "the future will be the same as the past." you aren't predicting anything. you will be wrong every single time something novel occurs, which is pretty frequently in the real world.
    - ryanwaggoner 9y ago
      The perception that weather forecasting is notoriously unreliable is mostly false: https://mobile.nytimes.com/2012/09/09/magazine/the-weatherman-is-not-a-moron.html?referer= https://mobile.nytimes.com/2012/09/09/magazine/the-weatherma...
      - sgt101 9y ago
        From your link : "Why are weather forecasters succeeding when other predictors fail? It’s because long ago they came to accept the imperfections in their knowledge. That helped them understand that even the most sophisticated computers, combing through seemingly limitless data, are painfully ill equipped to predict something as dynamic as weather all by themselves. So as fields like economics began relying more on Big Data, meteorologists recognized that data on its own isn’t enough."
        gbrown 9y ago
        Quantifying uncertainty is one of the main points of statistics. Don't confuse the limitations of point estimates provided by machine learning techniques with all of statistical practice.