5 ms·

Consider this set of statistics: 90% of elementary school teachers are female [1]. 90% of veterans are male [2]. Unsurprising. But, there are 2 million female

Consider this set of statistics: 90% of elementary school teachers are female [1]. 90% of veterans are male [2]. Unsurprising. But, there are 2 million female veterans in the US, and only 1.7 million female elementary school teachers. That means if you talk to a random American female, it is more likely that she is a veteran than an elementary school teacher. I think a lot of disagreements on Hacker News and elsewhere stem from people saying "Well 90% of the time, X is true" without realizing that the 10% they are choosing to ignore is comprised of millions of people. [1] https://nces.ed.gov/programs/coe/indicator_clr.asp https://nces.ed.gov/programs/coe/indicator_clr.asp [2] https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.va.gov/vetdata/docs/SpecialReports/Women_Veterans_2015_Final.pdf&ved=2ahUKEwjmkMashv_oAhURrp4KHaUHDwIQFjABegQIDRAG&usg=AOvVaw1HeRlMh4mSIEx0AaXjN_G7 https://www.google.com/url?sa=t&source=web&rct=j&url=https:/...

abrahamneben 7y ago
Enter bayes theorem
- sudosysgen 7y ago
  Bayes' Theorem should be getting taught with quite some emphasis in high school, imo.
  - larrydag 7y ago
    I would argue that Bayes Theorem is taught in high school basically conditional probabilities and inferences. The issue is that applied Bayes Theorem is not taught. What do you do with information given with the data at hand?
    - TeMPOraL 7y ago
      Bayesian updates should be taught in schools, at least at the most coarse-grained level. Not to let people calculate anything; almost nobody is going to need that in life anyway. But so that some authority (which a school is) tells people they're allowed and supposed to update their beliefs on evidence. That evidence drags belief more in one direction than the other. That 0 and 1 are not probabilities in real life. That being certain of something is a rare thing, that they should embrace being more or less sure. That this is not someone's random worldview, but there is a proper (and rather fundamental) mathematical formalism behind it, it's just impossible to apply it fully in real life, so we have to approximate it.
noname120 7y ago
This is called Simpson's paradox[1]. [1] https://en.wikipedia.org/wiki/Simpson%27s_paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox
- BeetleB 7y ago
  It's not an example of Simpson's Paradox. It's an example of the base rate fallacy. https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy
pmiller2 7y ago
> ...the 10% they are choosing to ignore is comprised of millions of people. I don't understand your point here. Is it that somehow, some people are more than 10% likely to run into the 10%? Well, that's obvious. Roughly speaking, you'd expect 10% of commenters to be in that 10%, unless we're talking about something that disproportionately does or does not affect people interested in technologies, startups, etc..
- MyHypatia 7y ago
  The point is that people often have an opinion, cite one statistic to validate their opinion, miss the bigger picture (because summarizing anything complex with a single number, rarely tells the whole story), and in the face of additional evidence stick to that one statistic that validates their original opinion. I see it happen a lot. Both on HN and elsewhere too. Presenting multiple statistics that shine a different light on the same population, enriches the conversation, and gives a more complete characterization of the population/situation/phenomena etc. I also personally found that combined set of statistics enlightening because I previously did not realize how large the population of female veterans is, since I had only read about the percentages not the absolute numbers.
  - TeMPOraL 7y ago
    It's a problem of lack of STEM education, unfortunately. More specifically, of metrology. "90% of elementary school teachers" is not directly comparable to "90% of veterans". You have to multiply them by their respective populations to get numbers that can be compared. In general, a thing of the form "$amount of $something" is not comparable to "$amount of $somethingelse". That amount being a fraction with a '%' sign in front of it doesn't change anything. Unfortunately, schools aren't beating people with sticks until they internalize that point, while at the same time a good part of sales and marketing relies on people not being formal enough in their thinking.
    - MyHypatia 7y ago
      Yes, I certainly wish schools would teach this better. I think showing different examples with surprising conclusions/outcomes helps people internalize that point.
Natsu 7y ago
That's because you need to know that there are ~20 million veterans and ~1.87 million elementary school teachers, i.e. you need to know the base rate if I'm reversing the calculations properly. Another flaw of averages is apparent when you realize that the average person has less than two eyes.
- noizejoy 7y ago
  And half of the population is dumber than the average. :-)
  - MR4D 7y ago
    Shouldn't it be: Half of the population is dumber than the median. ??
    - Natsu 7y ago
      This is true by default, the other depends on the distribution.
      - the8472 7y ago
        Since intelligence must be a one-sided distribution it's probably right-tailed, which means the mean is higher than the median. So I expect (slightly) more than half of the population to be dumber than average.
        alasdair_ 7y ago
        >Since intelligence must be a one-sided distribution it's probably right-tailed, which means the mean is higher than the median. There are a lot of people with physical injuries or disabilities that skew things to the left.
        the8472 7y ago
        Those are all small, independent random contributions that naively would lead to a normal distribution per the central limit theorem. But it can't be quite a normal distribution due to the lower bound of 0 (in terms of absolute intelligence, not IQ). A better candidate would be a log-normal distribution which is always right-skewed.
pyuser583 7y ago
This is midly-nitpicky, but those numbers only include public elementary school teachers. There are a lot of private/parochial school elementary teachers out there. There might very well be more female elementary school teachers then female veterans.

abrahamneben 7y ago

Enter bayes theorem

sudosysgen 7y ago
Bayes' Theorem should be getting taught with quite some emphasis in high school, imo.
- larrydag 7y ago
  I would argue that Bayes Theorem is taught in high school basically conditional probabilities and inferences. The issue is that applied Bayes Theorem is not taught. What do you do with information given with the data at hand?
  - TeMPOraL 7y ago
    Bayesian updates should be taught in schools, at least at the most coarse-grained level. Not to let people calculate anything; almost nobody is going to need that in life anyway. But so that some authority (which a school is) tells people they're allowed and supposed to update their beliefs on evidence. That evidence drags belief more in one direction than the other. That 0 and 1 are not probabilities in real life. That being certain of something is a rare thing, that they should embrace being more or less sure. That this is not someone's random worldview, but there is a proper (and rather fundamental) mathematical formalism behind it, it's just impossible to apply it fully in real life, so we have to approximate it.

noname120 7y ago

This is called Simpson's paradox[1]. [1] https://en.wikipedia.org/wiki/Simpson%27s_paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox

BeetleB 7y ago
It's not an example of Simpson's Paradox. It's an example of the base rate fallacy. https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy

pmiller2 7y ago

> ...the 10% they are choosing to ignore is comprised of millions of people. I don't understand your point here. Is it that somehow, some people are more than 10% likely to run into the 10%? Well, that's obvious. Roughly speaking, you'd expect 10% of commenters to be in that 10%, unless we're talking about something that disproportionately does or does not affect people interested in technologies, startups, etc..

MyHypatia 7y ago
The point is that people often have an opinion, cite one statistic to validate their opinion, miss the bigger picture (because summarizing anything complex with a single number, rarely tells the whole story), and in the face of additional evidence stick to that one statistic that validates their original opinion. I see it happen a lot. Both on HN and elsewhere too. Presenting multiple statistics that shine a different light on the same population, enriches the conversation, and gives a more complete characterization of the population/situation/phenomena etc. I also personally found that combined set of statistics enlightening because I previously did not realize how large the population of female veterans is, since I had only read about the percentages not the absolute numbers.
- TeMPOraL 7y ago
  It's a problem of lack of STEM education, unfortunately. More specifically, of metrology. "90% of elementary school teachers" is not directly comparable to "90% of veterans". You have to multiply them by their respective populations to get numbers that can be compared. In general, a thing of the form "$amount of $something" is not comparable to "$amount of $somethingelse". That amount being a fraction with a '%' sign in front of it doesn't change anything. Unfortunately, schools aren't beating people with sticks until they internalize that point, while at the same time a good part of sales and marketing relies on people not being formal enough in their thinking.
  - MyHypatia 7y ago
    Yes, I certainly wish schools would teach this better. I think showing different examples with surprising conclusions/outcomes helps people internalize that point.

Natsu 7y ago

That's because you need to know that there are ~20 million veterans and ~1.87 million elementary school teachers, i.e. you need to know the base rate if I'm reversing the calculations properly. Another flaw of averages is apparent when you realize that the average person has less than two eyes.

noizejoy 7y ago
And half of the population is dumber than the average. :-)
- MR4D 7y ago
  Shouldn't it be: Half of the population is dumber than the median. ??
  - Natsu 7y ago
    This is true by default, the other depends on the distribution.
    - the8472 7y ago
      Since intelligence must be a one-sided distribution it's probably right-tailed, which means the mean is higher than the median. So I expect (slightly) more than half of the population to be dumber than average.
      - alasdair_ 7y ago
        >Since intelligence must be a one-sided distribution it's probably right-tailed, which means the mean is higher than the median. There are a lot of people with physical injuries or disabilities that skew things to the left.
        the8472 7y ago
        Those are all small, independent random contributions that naively would lead to a normal distribution per the central limit theorem. But it can't be quite a normal distribution due to the lower bound of 0 (in terms of absolute intelligence, not IQ). A better candidate would be a log-normal distribution which is always right-skewed.

pyuser583 7y ago

This is midly-nitpicky, but those numbers only include public elementary school teachers. There are a lot of private/parochial school elementary teachers out there. There might very well be more female elementary school teachers then female veterans.