4 ms·
Good opportunity to plug https://en.wikipedia.org/wiki/Anscombe%27s_quartet https://en.wikipedia.org/wiki/Anscombe%27s_quartet : if you don't know much about th
by madars 5y ago
Good opportunity to plug https://en.wikipedia.org/wiki/Anscombe%27s_quartet https://en.wikipedia.org/wiki/Anscombe%27s_quartet : if you don't know much about the underlying distribution, simple statistics don't describe it well.
From Wikipedia description: Anscombe's quartet comprises four data sets that have nearly identical simple descriptive statistics, yet have very different distributions and appear very different when graphed. Each dataset consists of eleven (x,y) points. They were constructed in 1973 by the statistician Francis Anscombe to demonstrate both the importance of graphing data before analyzing it, and the effect of outliers and other influential observations on statistical properties.
- djk447 5y agoNB: Post author here. That's really nifty, wish I'd heard about it earlier. Might go back and add a link to it in the post at some point too! Very useful. Definitely know I wasn't breaking new ground or anything, but fun to see it represented so succinctly.
- doctorsher 5y agoThis is excellent information, thank you for posting this! I was not familiar with this example previously, but it is a perfect example of summary statistics not capturing certain distributions well. It's very approachable, even if you had to limit the discussion to mean and variance alone. Bookmarked, and much appreciated.
- pdpi 5y agoThere's a fun paper by Autodesk where they make datasets that look whatever way you want them to. https://www.autodesk.com/research/publications/same-stats-different-graphs https://www.autodesk.com/research/publications/same-stats-di...
- djk447 5y agoNB: Post author here. This is great! So fun...will have to use in the future...
- pdpi 5y agoWhile I have your attention... > For the median and average to be equal, the points less than the median and greater than the median must have the same distribution (i.e., there must be the same number of points that are somewhat larger and somewhat smaller and much larger and much smaller). [0, 2, 5, 9, 9] has both median and mean = 5, but the two sides don't really have the same distribution.
- djk447 5y agoTotally true...thoughts on how I could rephrase? I guess it's more the "weight" of points greater than and less than the median should be the same, so symmetric distributions definitely have it, asymmetric may or may not. Definitely open to revising...
- pdpi 5y ago> symmetric distributions definitely have it, asymmetric may or may not Doesn't have to be any more complicated than that. It's more a curio than an important point anyway :)
- s_gourichon 5y agoYes, a both funny and insightful lesson on how weak basic indicators (mean, standard deviation, correlation) can be, the interest and limits of box-plot with quartiles and whiskers, the benefit of the violin-plot. Definitely worth a quick look.
- deleted 5y ago[deleted]
- potatoman22 5y agoInteresting, thanks for the concept. What's one to do for high dimensional data then?
- hoseja 5y ago>importance of graphing data before analyzing it Very discouraging if one is trying to analyze data algorithmically. Often when faced with a problem in statistics, the answer is: "Look at the graph and use intuition!".