4 ms·

>I could be way off, but I am confident of this assessment. Having been briefly spoiled by the way some amateur analysts like Scott Alexander use percentages t

>I could be way off, but I am confident of this assessment. Having been briefly spoiled by the way some amateur analysts like Scott Alexander use percentages to express their confidence, reading that sentence was like stepping out of a warm room into the cold outside air.

aerovistae 5y ago
I literally have no idea what this comment means. I re-read it several times unsuccessfully. I am familiar with Scott Alexander but that did not help my comprehension.
- KevinGlass 5y ago
  The parent commenter is which the author of this blog post expressed their confidence in this analysis as a percentage, i.e. I'm 65% sure that Russia will lose; or I give Russia a 65% change of winning. That way it's easy to compare different opinions and see who was "more right".
  - maw 5y ago
    It gives actual insight into prognosticators' accuracy. If their 90%s are right about nine times out of ten, their 60%s six out of ten, and so on, then they're pretty good. If they're not then they're not.
    - smitty1e 5y ago
      Subjective certainty is not objective accuracy. Throwing out percentages without methodology and test data bolstering that methodology amounts to persuasion.
      - whatshisface 5y ago
        A prediction with no indication of subjective certainty is not a prediction.
  - aerovistae 5y ago
    I sort of lost faith in percentage guesses the night of the 2016 US elections when I watched the NYTimes prediction flip from >95% Clinton wins to >99% Trump wins in a matter of seconds. It really opened my eyes to how illusory the utility of the percentage was - if it can flip that hard that fast, clearly it was never really only a <5% chance of happening, and the guesser really didn't know anything meaningful. It's astrology for the educated.
    - whatshisface 5y ago
      If bad predictions make you lose faith in percentages as a whole, why don't lies make you lose faith in words as a whole? In both cases I would assign the fault with the errant speaker.
    - anothernewdude 5y ago
      Because their predictions were based on how correct polling was, and once they found gross errors in the polling they adjusted.
    - Ariarule 5y ago
      While the change in who was favored to win the 2016 election was indeed large, it didn't change that dramatically and didn't change in seconds. The NYT 2016 prediction chart is still available on their site: https://www.nytimes.com/elections/2016/forecast/president https://www.nytimes.com/elections/2016/forecast/president The labeling on the horizontal axis on the chart is terrible, but roughly this is a change from 20% to 95% in about 2 or 3 hours. It doesn't look like they ever had it at less than 15% (again, give or take a few points due to bad infographic-esque picture).
    - wolverine876 5y ago
      > I watched the NYTimes prediction flip from >95% Clinton wins to >99% Trump wins in a matter of seconds. Did that really happen? The numbers I remember before the election were about 67% chance of Clinton, though I don't recall the NYT. On election night, I recall Clinton not doing well from the start.
zwkrt 5y ago
It has its positives and negatives. It’s pretty misleading to couch a prediction about a one time event in percentage terms; remember 538’s confidence that Hillary would win the election in 2016?
- tenuousemphasis 5y ago
  You do realize an event with a 79% chance of happening still doesn't happen 1 out of 5 times? That doesn't mean that the 79% prediction is wrong, that's just how statistics work.
- LordDragonfang 5y ago
  Interesting choice of example, because 538 actually gave the least confidence of any poll analysis/reporting site. https://fivethirtyeight.com/features/why-fivethirtyeight-gave-trump-a-better-chance-than-almost-anyone-else/ https://fivethirtyeight.com/features/why-fivethirtyeight-gav...
- whatshisface 5y ago
  If I minted a well-balanced coin and flipped it with a promise to destroy it afterwards, it would not at all be misleading to characterize it as having a 50/50 chance of coming up heads. Even in the frequentist interpretation of probability, the odds before a one-time event have a meaning: they're the fraction of similar events that would resolve in a certain way if it was to be repeated. >remember 538’s confidence that Hillary would win the election in 2016 Yeah, 538 was confident. It wasn't misleading for them to announce a high probability, unless you thought 538's confidence was the same thing as The Probability.
richliss 5y ago
That quote has Sex Panther vibes.

aerovistae 5y ago

I literally have no idea what this comment means. I re-read it several times unsuccessfully. I am familiar with Scott Alexander but that did not help my comprehension.

KevinGlass 5y ago
The parent commenter is which the author of this blog post expressed their confidence in this analysis as a percentage, i.e. I'm 65% sure that Russia will lose; or I give Russia a 65% change of winning. That way it's easy to compare different opinions and see who was "more right".
- maw 5y ago
  It gives actual insight into prognosticators' accuracy. If their 90%s are right about nine times out of ten, their 60%s six out of ten, and so on, then they're pretty good. If they're not then they're not.
  - smitty1e 5y ago
    Subjective certainty is not objective accuracy. Throwing out percentages without methodology and test data bolstering that methodology amounts to persuasion.
    - whatshisface 5y ago
      A prediction with no indication of subjective certainty is not a prediction.
- aerovistae 5y ago
  I sort of lost faith in percentage guesses the night of the 2016 US elections when I watched the NYTimes prediction flip from >95% Clinton wins to >99% Trump wins in a matter of seconds. It really opened my eyes to how illusory the utility of the percentage was - if it can flip that hard that fast, clearly it was never really only a <5% chance of happening, and the guesser really didn't know anything meaningful. It's astrology for the educated.
  - whatshisface 5y ago
    If bad predictions make you lose faith in percentages as a whole, why don't lies make you lose faith in words as a whole? In both cases I would assign the fault with the errant speaker.
  - anothernewdude 5y ago
    Because their predictions were based on how correct polling was, and once they found gross errors in the polling they adjusted.
  - Ariarule 5y ago
    While the change in who was favored to win the 2016 election was indeed large, it didn't change that dramatically and didn't change in seconds. The NYT 2016 prediction chart is still available on their site: https://www.nytimes.com/elections/2016/forecast/president https://www.nytimes.com/elections/2016/forecast/president The labeling on the horizontal axis on the chart is terrible, but roughly this is a change from 20% to 95% in about 2 or 3 hours. It doesn't look like they ever had it at less than 15% (again, give or take a few points due to bad infographic-esque picture).
  - wolverine876 5y ago
    > I watched the NYTimes prediction flip from >95% Clinton wins to >99% Trump wins in a matter of seconds. Did that really happen? The numbers I remember before the election were about 67% chance of Clinton, though I don't recall the NYT. On election night, I recall Clinton not doing well from the start.

zwkrt 5y ago

It has its positives and negatives. It’s pretty misleading to couch a prediction about a one time event in percentage terms; remember 538’s confidence that Hillary would win the election in 2016?

tenuousemphasis 5y ago
You do realize an event with a 79% chance of happening still doesn't happen 1 out of 5 times? That doesn't mean that the 79% prediction is wrong, that's just how statistics work.
LordDragonfang 5y ago
Interesting choice of example, because 538 actually gave the least confidence of any poll analysis/reporting site. https://fivethirtyeight.com/features/why-fivethirtyeight-gave-trump-a-better-chance-than-almost-anyone-else/ https://fivethirtyeight.com/features/why-fivethirtyeight-gav...
whatshisface 5y ago
If I minted a well-balanced coin and flipped it with a promise to destroy it afterwards, it would not at all be misleading to characterize it as having a 50/50 chance of coming up heads. Even in the frequentist interpretation of probability, the odds before a one-time event have a meaning: they're the fraction of similar events that would resolve in a certain way if it was to be repeated. >remember 538’s confidence that Hillary would win the election in 2016 Yeah, 538 was confident. It wasn't misleading for them to announce a high probability, unless you thought 538's confidence was the same thing as The Probability.

richliss 5y ago

That quote has Sex Panther vibes.