5 ms·

This effect has a name, The Simpsons paradox [0] The graph in [1] explains it much better than any words could. [0] https://en.wikipedia.org/wiki/Simpson%27s_

• JadeNB 7y ago
  > This effect has a name, The Simpsons paradox [0] No relation to The Simpsons; it is Simpson's paradox, after Edward Simpson (e.g., https://ftp.cs.ucla.edu/pub/stat_ser/r414.pdf https://ftp.cs.ucla.edu/pub/stat_ser/r414.pdf).
• xingped 7y ago
  That was a fascinating read, thanks!
• sbierwagen 7y ago
  https://imgur.com/a/lg3gwPJ https://imgur.com/a/lg3gwPJ Not sure about the ad placement in that analyticsindiamag.com article.
• allenu 7y ago
  Talk about easily misreading a graph... I thought the article in the second link was going to explain the decline of the Simpsons based on frequency of episodes focusing on specific characters, but it was just trying to be clever with that image.
• marzell 7y ago
  Seems to be the statistics equivalent of gerrymandering.
  • abtinf 7y ago
    Gerrymandering is an intentional act to achieve specific outcome. Simpson's Paradox can strike even the most committed truth-seeker trying to understand and interpret the available data. Edit to add: Simpson's Paradox could conceivably thwart a gerrymanderer if they have an erroneous model relating demographics to voting patterns.
    • marzell 7y ago
      This is all true.. However what I meant is that the way gerrymandering can indicate a winner that is not the majority choice, is quasi analogous to how the Simpson's paradox exemplifies a weighed average that disagrees with the subtotals of the data.
      • abtinf 7y ago
        Oh, I see—you mean from the perspective of the voter/pundit/analyst/politician trying to understand what an election means and their mandate for action. Sort of like the debate over electoral vs popular vote. I hadn’t thought about it that way or seen the word gerrymandering used in that way, but it makes sense.
  • Tarq0n 7y ago
    One interesting aspect to Simpson's paradox it that it's not straightforward which perspective is the more valid one, by groups or combined. One example in which the combined perspective is the valid one is election results (it is possible to win a higher percentage of votes in multiple areas, yet lose the overall vote).
• Tarq0n 7y ago
  That's part of it, but drawing a linear fit through the data shown in the article is outrageously misleading in itself, and should be rejected outright.
  • xenocyon 7y ago
    There's nothing inherently wrong with a linear fit and a linear model is often more accurate than a more ostensibly correct-looking polynomial or complex function which overfits the observed data. Perhaps you mean that drawing any kind of fit though this data is misleading, when lacking confidence statistics and R2 score - although you could have all those and still do poorly on held-out data, or you could even do well on held-out data and still be victim to confounders or misunderstanding the direction of causation... I see a line through data as nothing more than a hypothesis, which needs to be backed up with rigor. If it's backed up, there's nothing outrageous about it.
    • Tarq0n 7y ago
      The residuals of this model look really bad though, and that should tell you that there's insufficient predictive value. I'd at a minimum use a piecewise fir, or simply not fit a model without further predictors.
• xenocyon 7y ago
  Not quite; Simpson's paradox is when splitting a group into subgroups reverses the direction of correlation in every subgroup of the split, which is not the case in the example presented here. Aside: it's not always true that the direction of correlation in the sliced groups is more correct than that of the larger group. You need causal analysis to know when it is incorrect or correct to slice the data to eliminate confounders.