6 ms·

68–95–99.7 Rule

131 points by turrini 4y ago

abakker 4y ago
I always like the inverse of this, which is that if you calculate that less than 68% is in the first standard deviation, you can guess that the data isn’t normally distributed. It’s really a heuristic on where to start with analysis, though. Not a result in and of itself.
- qsort 4y ago
  The gist of the idea is of course correct, but if you actually do this, please use an actual normality test. https://en.wikipedia.org/wiki/Shapiro–Wilk_test https://en.wikipedia.org/wiki/Shapiro–Wilk_test https://en.wikipedia.org/wiki/Anderson–Darling_test https://en.wikipedia.org/wiki/Anderson–Darling_test https://en.wikipedia.org/wiki/Kolmogorov–Smirnov_test https://en.wikipedia.org/wiki/Kolmogorov–Smirnov_test
  - disgruntledphd2 4y ago
    The trouble with normality tests is that you either have too little data (in which case they're useless) or too much (in which case their also useless). I always think of normality as a nice theory for simple stuff, but you want to be running sensitivity analyses for all of your assumptions, including normality (which I don't think I've ever seen professionally, tbh).
paulpauper 4y ago
This is why also claims of IQs above 180 or so are almost always BS, at least on normed tests. 6sd is about the maximum
- ben_w 4y ago
  Anything outside 70-130 is suspect in practice, as the tests aren't well calibrated beyond ±2σ. I'm not sure what intellectual performance is even supposed to be implied by any given IQ score, given that the scores commonly used today are just a linear mapping from standard deviations above or below mean. That said, on the low end at -3σ I would expect a serious deviation from a normal distribution just from the effect of dementia.
  - qsort 4y ago
    Sure, but the point is that 180 wouldn't make sense even if the test were perfect. There aren't enough humans on earth for that many standard deviations to exist in the first place. 180 IQ could as well be 2000 IQ, the sky is the limit when the scale makes no sense.
    - computerphage 4y ago
      That's just wrong? 6 sigma would be IQ 100 + 6 * 15 = 190. 6 sigma is 1 in 506,797,346. So there should be 16 people with IQs that extreme if intelligence is normally distributed. (There's a factor of two somewhere in here for the two tailed nature, but the point stands)
    - delecti 4y ago
      Either I'm misunderstanding the math, or you are. An IQ of 180 is 5 1/3 SD above mean, which is a bit more frequent than 1 in 26,330,254 (which would be 5.5 SD). Half that common because we're only talking about above, and 1 in 50m people on earth is still 160 people. That's not many, but I don't think it's negligible either. https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...
- scythmic_waves 4y ago
  It's also literally impossible for many modern tests. For example the WAIS-IV, a very popular measure, has a maximum possible FSIQ of 160. With a mean of 100 and an SD of 15, that's "only" 4 SDs from the mean (99.997 percentile). And as you say, modern tests just aren't calibrated in that way. The WAIS-IV is calibrated for the 70-130 range [0]. It's used to provide broad ranges (e.g. 85-100 vs. 115-130), not to provide useful differentiations on the high end like 158 vs. 159. In fact, the error increases with the score [1]. There is less certainty for a score of 145 than there is for 115. It's just not designed for separating geniuses from super geniuses or however you want to put it. [0] https://www.quora.com/Whats-the-highest-IQ-score-you-can-get-on-the-WAIS-IV https://www.quora.com/Whats-the-highest-IQ-score-you-can-get... (Not a direct source I'm afraid, but the materials are proprietary.) [1] https://en.wikipedia.org/wiki/IQ_classification#Giftedness https://en.wikipedia.org/wiki/IQ_classification#Giftedness (There are several citations in this section, pertinent discussion below the table.)
H8crilA 4y ago
Remember that a lot of data is not actually normally distributed, even though it looks so at first. The horror of "risk management" units of financial institutions, when they get a "7-sigma" event twice in a month. Can you imagine there was a time when the entire US options market was running on flat volatility vs strike price (implying that financial asset prices are lognormal). Must have been cool to just buy the cheapest, most OTM options and collect big results every now and then.
- bombcar 4y ago
  Also if it’s once in 300 million to happen every day, it could happen to someone every day in the USA (assuming it’s something people do daily). If the outfall from a 7-sigma is disastrous, you have to try to prevent it, because it WILL happen eventually given enough goes.
  - H8crilA 4y ago
    That's true but those people were sampling closer to once per day, and still got N-sigma in their models for some high value of N. The easiest way to get something that's not actually normal in the typical central limit theorem construction is to have the variables be just slightly not independent. Which becomes obvious in retrospect after you start realizing the dependence and shit hits the fan.
    - bombcar 4y ago
      Banks failing and CDOs exploding would be two good examples; they’re rare if not nonexistent but if they start they may all flare up. Also natural distribution assumes you don’t have someone trying to game the extremes to make money. If being seven feet tall resulted in billions we’d have people researching how to grow taller people. You can see stuff like this in distributions of tax returns - many people will be unexpectedly clustered right below certain cut-off amounts.
      - looping__lui 4y ago
djaychela 4y ago
The table of numerical values is pretty interesting for a layman. I've often heard scientists (such as Brian Cox) talk about x-sigma probabilities, but putting it the context in this table is much more meaningful for someone like me who only has self-study reading as scientific understanding.
tedunangst 4y ago
Memorizing a set of numbers doesn't really feel like a shorthand way to remember those numbers. Did you know about the 3.14 rule? It's a way to remember the first three digits of pi are 3.14.
- sroussey 4y ago
  Which is, I don’t know — ironic? — because pi rounded to two decimals is 3.15.
  - elijaht 4y ago
    No it’s not- it’s 3.141, which rounds to 3.14
    - midasuni 4y ago
      It’s 3.14159 3.1416 3.142 3.14 3.1 3
      - sroussey 4y ago
        Oh, I feel dumb. Some reason I had it memorized without the 1: 3.1459 — for years! Decades! I wonder what tests I screwed up in collage!
  - scythmic_waves 4y ago
    No pi is 3.14159... which is 3.14 when rounded.
  - karmakaze 4y ago
    The 3.2 rounding of Pi in an Indiana bill[0] is also quite infamous, which confusingly had other values of Pi for use in different contexts. [I'm a Tau touter myself.] [0] https://www.straightdope.com/21341975/did-a-state-legislature-once-pass-a-law-saying-pi-equals-3 https://www.straightdope.com/21341975/did-a-state-legislatur...
  - euroderf 4y ago
Eddy_Viscosity2 4y ago
I like how they call this a 'rule' instead just a list of three hard to remember numbers in a row.
begemotz 4y ago
Mentioned in the OP, but perhaps buried a bit, I am always partial to Chebyshev's Inequality (https://en.wikipedia.org/wiki/Chebyshev%27s_inequality https://en.wikipedia.org/wiki/Chebyshev%27s_inequality) since it describes all distributions not just Gaussian. While approximations, they are nice to have when you question the normality of the data and still want to estimate limits - more numbers to remember however, since they are estimates.

131 points by turrini 4y ago

I always like the inverse of this, which is that if you calculate that less than 68% is in the first standard deviation, you can guess that the data isn’t normally distributed. It’s really a heuristic on where to start with analysis, though. Not a result in and of itself.

qsort 4y ago
The gist of the idea is of course correct, but if you actually do this, please use an actual normality test. https://en.wikipedia.org/wiki/Shapiro–Wilk_test https://en.wikipedia.org/wiki/Shapiro–Wilk_test https://en.wikipedia.org/wiki/Anderson–Darling_test https://en.wikipedia.org/wiki/Anderson–Darling_test https://en.wikipedia.org/wiki/Kolmogorov–Smirnov_test https://en.wikipedia.org/wiki/Kolmogorov–Smirnov_test
- disgruntledphd2 4y ago
  The trouble with normality tests is that you either have too little data (in which case they're useless) or too much (in which case their also useless). I always think of normality as a nice theory for simple stuff, but you want to be running sensitivity analyses for all of your assumptions, including normality (which I don't think I've ever seen professionally, tbh).

paulpauper 4y ago

This is why also claims of IQs above 180 or so are almost always BS, at least on normed tests. 6sd is about the maximum

ben_w 4y ago
Anything outside 70-130 is suspect in practice, as the tests aren't well calibrated beyond ±2σ. I'm not sure what intellectual performance is even supposed to be implied by any given IQ score, given that the scores commonly used today are just a linear mapping from standard deviations above or below mean. That said, on the low end at -3σ I would expect a serious deviation from a normal distribution just from the effect of dementia.
- qsort 4y ago
  Sure, but the point is that 180 wouldn't make sense even if the test were perfect. There aren't enough humans on earth for that many standard deviations to exist in the first place. 180 IQ could as well be 2000 IQ, the sky is the limit when the scale makes no sense.
  - computerphage 4y ago
    That's just wrong? 6 sigma would be IQ 100 + 6 * 15 = 190. 6 sigma is 1 in 506,797,346. So there should be 16 people with IQs that extreme if intelligence is normally distributed. (There's a factor of two somewhere in here for the two tailed nature, but the point stands)
  - delecti 4y ago
    Either I'm misunderstanding the math, or you are. An IQ of 180 is 5 1/3 SD above mean, which is a bit more frequent than 1 in 26,330,254 (which would be 5.5 SD). Half that common because we're only talking about above, and 1 in 50m people on earth is still 160 people. That's not many, but I don't think it's negligible either. https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...
scythmic_waves 4y ago
It's also literally impossible for many modern tests. For example the WAIS-IV, a very popular measure, has a maximum possible FSIQ of 160. With a mean of 100 and an SD of 15, that's "only" 4 SDs from the mean (99.997 percentile). And as you say, modern tests just aren't calibrated in that way. The WAIS-IV is calibrated for the 70-130 range [0]. It's used to provide broad ranges (e.g. 85-100 vs. 115-130), not to provide useful differentiations on the high end like 158 vs. 159. In fact, the error increases with the score [1]. There is less certainty for a score of 145 than there is for 115. It's just not designed for separating geniuses from super geniuses or however you want to put it. [0] https://www.quora.com/Whats-the-highest-IQ-score-you-can-get-on-the-WAIS-IV https://www.quora.com/Whats-the-highest-IQ-score-you-can-get... (Not a direct source I'm afraid, but the materials are proprietary.) [1] https://en.wikipedia.org/wiki/IQ_classification#Giftedness https://en.wikipedia.org/wiki/IQ_classification#Giftedness (There are several citations in this section, pertinent discussion below the table.)

H8crilA 4y ago

Remember that a lot of data is not actually normally distributed, even though it looks so at first. The horror of "risk management" units of financial institutions, when they get a "7-sigma" event twice in a month. Can you imagine there was a time when the entire US options market was running on flat volatility vs strike price (implying that financial asset prices are lognormal). Must have been cool to just buy the cheapest, most OTM options and collect big results every now and then.

bombcar 4y ago
Also if it’s once in 300 million to happen every day, it could happen to someone every day in the USA (assuming it’s something people do daily). If the outfall from a 7-sigma is disastrous, you have to try to prevent it, because it WILL happen eventually given enough goes.
- H8crilA 4y ago
  That's true but those people were sampling closer to once per day, and still got N-sigma in their models for some high value of N. The easiest way to get something that's not actually normal in the typical central limit theorem construction is to have the variables be just slightly not independent. Which becomes obvious in retrospect after you start realizing the dependence and shit hits the fan.
  - bombcar 4y ago
    Banks failing and CDOs exploding would be two good examples; they’re rare if not nonexistent but if they start they may all flare up. Also natural distribution assumes you don’t have someone trying to game the extremes to make money. If being seven feet tall resulted in billions we’d have people researching how to grow taller people. You can see stuff like this in distributions of tax returns - many people will be unexpectedly clustered right below certain cut-off amounts.
    - looping__lui 4y ago

djaychela 4y ago

The table of numerical values is pretty interesting for a layman. I've often heard scientists (such as Brian Cox) talk about x-sigma probabilities, but putting it the context in this table is much more meaningful for someone like me who only has self-study reading as scientific understanding.

tedunangst 4y ago

Memorizing a set of numbers doesn't really feel like a shorthand way to remember those numbers. Did you know about the 3.14 rule? It's a way to remember the first three digits of pi are 3.14.

sroussey 4y ago
Which is, I don’t know — ironic? — because pi rounded to two decimals is 3.15.
- elijaht 4y ago
  No it’s not- it’s 3.141, which rounds to 3.14
  - midasuni 4y ago
    It’s 3.14159 3.1416 3.142 3.14 3.1 3
    - sroussey 4y ago
      Oh, I feel dumb. Some reason I had it memorized without the 1: 3.1459 — for years! Decades! I wonder what tests I screwed up in collage!
- scythmic_waves 4y ago
  No pi is 3.14159... which is 3.14 when rounded.
- karmakaze 4y ago
  The 3.2 rounding of Pi in an Indiana bill[0] is also quite infamous, which confusingly had other values of Pi for use in different contexts. [I'm a Tau touter myself.] [0] https://www.straightdope.com/21341975/did-a-state-legislature-once-pass-a-law-saying-pi-equals-3 https://www.straightdope.com/21341975/did-a-state-legislatur...
- euroderf 4y ago

Eddy_Viscosity2 4y ago

I like how they call this a 'rule' instead just a list of three hard to remember numbers in a row.

begemotz 4y ago

Mentioned in the OP, but perhaps buried a bit, I am always partial to Chebyshev's Inequality (https://en.wikipedia.org/wiki/Chebyshev%27s_inequality https://en.wikipedia.org/wiki/Chebyshev%27s_inequality) since it describes all distributions not just Gaussian. While approximations, they are nice to have when you question the normality of the data and still want to estimate limits - more numbers to remember however, since they are estimates.