4 ms·

It's just probability theory meets survivorship bias. The larger the team the more it will trend towards the average because the joint probability that everyone

It's just probability theory meets survivorship bias. The larger the team the more it will trend towards the average because the joint probability that everyone in the team is simultaneously awesome (or awful) becomes lower and lower as a function of team size. Same thing happened with "small schools perform better" so they got funded. But turns out it's possible for them to perform terribly for the same reason it is for them to perform well.

still_grokking 4y ago
> Same thing happened with "small schools perform better" so they got funded. But turns out it's possible for them to perform terribly for the same reason it is for them to perform well. And reason(s) would be?
- nullsense 4y ago
  Joint probability all members are either all excellent or all terrible is higher when the number of people is smaller.
  - still_grokking 4y ago
    Probability changes with sample size? Any source for that claim? AFAIK only variance changes with sample size. Otherwise statistics would be quite useless in general as I understand it. Say in the general population 10% are excellent. If I look at 10 randomly selected people I expect that one of them is excellent. If I look at 10 000 people should I really expect that more than 1 000 are excellent? That claim does not make sense to me.
    - misnome 4y ago
      > Probability changes with sample size? Any source for that claim? The post only suggests this if you ignore the word "Joint" before the word "probability". Otherwise, you seem to be arguing that the probability of getting "all heads" would be identical when flipping two coins as when flipping 100.
      - still_grokking 4y ago
        Even in smallish teams the probability that all members are excellent (or terrible) is almost zero, imho. In a bigger selection it's just closer to zero. In fact in any sample the distribution of "excellent, average, and terrible" will be more or less the same. Namely more or less like the distribution in the general population. (Otherwise we can stop looking at statistics in general as they would be worthless). The probability for the most unlikely event (namely a result that is maximally far away form the expected one) may be a little bit greater given smaller sample sizes (a result of broader variance), but this is irrelevant as the probability for such outcome is in almost any case infinitesimally small. Looking at the verges of a distribution to draw conclusions is misleading, imho. You need to look at the middle of the curve as there are the most likely outcomes of some selection. So, if I put some random people in a group the most likely outcome is that the ratio of excellent people to others is similar to the ratio of excellent people to other in the general population. This is the most likely outcome, independent of sample size.
        Jensson 4y ago
        Smart people often have smart friends, so a small team of mostly smart people isn't that unlikely. The problem is when you get a few degrees of separation, at that point the original signal is mostly lost and you will get the same kind of people as everyone else who uses the same hiring bar as you.
        still_grokking 4y ago
        This argument does not make much sense to me. If you pick only "the best" people by hand you could create a team with only excellent members of any size. All the statistical arguments become mot in such a scenario! You would get a team of only excellent people with a guarantied probability of 1, at any sample size.
        Jensson 4y ago
        Your argument doesn't work, nobody knows enough people to staff up a big company with only people they know. At some point they will have to go by surface signals or by word of mouth, talking to a person in an interview isn't enough. On the other hand for a small team, lets say 10 people, you could go back and pick the 10 best people you have worked with in your life. That will be a good team if you actually tries to get good people instead of people you like. That doesn't scale, other people will take people they like and quality suffers, it always happens at scale, but an individual kan make better choices.
    - nullsense 4y ago
      >AFAIK only variance changes with sample size. And what explains the change in variance as sample size changes?

still_grokking 4y ago

> Same thing happened with "small schools perform better" so they got funded. But turns out it's possible for them to perform terribly for the same reason it is for them to perform well. And reason(s) would be?

nullsense 4y ago
Joint probability all members are either all excellent or all terrible is higher when the number of people is smaller.
- still_grokking 4y ago
  Probability changes with sample size? Any source for that claim? AFAIK only variance changes with sample size. Otherwise statistics would be quite useless in general as I understand it. Say in the general population 10% are excellent. If I look at 10 randomly selected people I expect that one of them is excellent. If I look at 10 000 people should I really expect that more than 1 000 are excellent? That claim does not make sense to me.
  - misnome 4y ago
    > Probability changes with sample size? Any source for that claim? The post only suggests this if you ignore the word "Joint" before the word "probability". Otherwise, you seem to be arguing that the probability of getting "all heads" would be identical when flipping two coins as when flipping 100.
    - still_grokking 4y ago
      Even in smallish teams the probability that all members are excellent (or terrible) is almost zero, imho. In a bigger selection it's just closer to zero. In fact in any sample the distribution of "excellent, average, and terrible" will be more or less the same. Namely more or less like the distribution in the general population. (Otherwise we can stop looking at statistics in general as they would be worthless). The probability for the most unlikely event (namely a result that is maximally far away form the expected one) may be a little bit greater given smaller sample sizes (a result of broader variance), but this is irrelevant as the probability for such outcome is in almost any case infinitesimally small. Looking at the verges of a distribution to draw conclusions is misleading, imho. You need to look at the middle of the curve as there are the most likely outcomes of some selection. So, if I put some random people in a group the most likely outcome is that the ratio of excellent people to others is similar to the ratio of excellent people to other in the general population. This is the most likely outcome, independent of sample size.
      - Jensson 4y ago
        Smart people often have smart friends, so a small team of mostly smart people isn't that unlikely. The problem is when you get a few degrees of separation, at that point the original signal is mostly lost and you will get the same kind of people as everyone else who uses the same hiring bar as you.
        still_grokking 4y ago
        This argument does not make much sense to me. If you pick only "the best" people by hand you could create a team with only excellent members of any size. All the statistical arguments become mot in such a scenario! You would get a team of only excellent people with a guarantied probability of 1, at any sample size.
        Jensson 4y ago
        Your argument doesn't work, nobody knows enough people to staff up a big company with only people they know. At some point they will have to go by surface signals or by word of mouth, talking to a person in an interview isn't enough. On the other hand for a small team, lets say 10 people, you could go back and pick the 10 best people you have worked with in your life. That will be a good team if you actually tries to get good people instead of people you like. That doesn't scale, other people will take people they like and quality suffers, it always happens at scale, but an individual kan make better choices.
  - nullsense 4y ago
    >AFAIK only variance changes with sample size. And what explains the change in variance as sample size changes?