7 ms·

The diversity trumps ability theorem

maxklein 18y ago
In my experience, diversity in temperament is essential for any team to function. If you put two people who are highly excitable together, they will keep running after new ideas and feeding of each others excitement. If you put two optimists together, they will tend to gamble and play too risky. If you put a team of idea men together, you tend to have designs and nobody doing the actual work. I believe every team should be made up of a bunch of slow and steady chaps, and a few creative people, with the creative people split into excitable and careful. The guys that one often assumes are slow usually have really good solutions, but when the team is too fast paced, those ideas get drowned out by the people who are loud and confident of their own ability. Soon, the quiet chaps stop talking, because they know nobody will listen to them. Diversity has nothing to do with ethnicity if you are faced with a programming task. If you were to solve a socio-cultural problem, then diversity in terms of ethnicity would probably be a good division criteria.
michael_nielsen 18y ago
The punchline, from the abstract: "We find that when selecting a problem-solving team from a diverse population of intelligent agents, a team of randomly selected agents outperforms a team comprised of the best-performing agents. This result relies on the intuition that, as the initial pool of problem solvers becomes large, the best-performing agents necessarily become similar in the space of problem solvers. Their relatively greater ability is more than offset by their lack of problem-solving diversity." It's not always true, of course, that a randomly selected group will outperform a group of top performers. But the result does suggest that it's worth thinking hard about the tradeoff between diversity and ability, and that sometimes the former is more important than the latter.
DanielBMarkham 18y ago
Very insightful. Excellent catch.
thras 18y ago
Unlikely. This is a theoretical paper, not based on any actual data. What the authors get wrong is assuming that there is no correlation between intelligence and original thinking. It is an unwarranted and frankly stupid assumption. The claim that diversity trumps ability is defeated in nearly every real world test. I notice that the author's advice, to hire a random sample of candidates for the position instead of the best, is not followed by any great companies that I can think of. In fact, it's 180 degrees away from the hiring practices of places like Google, etc.
- maxklein 18y ago
  How many 'original thinkers' do you need in a company made up of 50.000 people? I'm sure you're more intelligent than I am, but don't you think there are some problems I can solve better than you, based off my past experience? Diverse usually implies diverse in experience, and not in intelligence.
- DanielBMarkham 18y ago
  It's not an either-or choice. Best-case is to hire a highly-intelligent, intellectually diverse team. Bingo Presto, life is good.
  - thras 18y ago
    Even then, I wonder whether maximizing intellectual diversity is really the best strategy. Teams with hefty philosophical disagreements can have trouble getting on the same page enough to accomplish goals. There may be an optimal level for intellectual diversity, but I wonder if it isn't closer to the minimum than the maximum for most tasks. It would be interesting to see real data on that; I have none.
    - DanielBMarkham 18y ago
      Yes. I think you have to add social skills into the mix. Ideally maximum intellectual diversity with maximum social (negotiation, facilitation, humility, leadership) abilities, but you'd have to scale back on intellectual diversity depending on the social ability of the group.
Eliezer 18y ago
Scott Page again? See here for a deconstruction of one of his previous, simpler arguments. http://www.overcomingbias.com/2007/04/the_error_of_cr.html http://www.overcomingbias.com/2007/04/the_error_of_cr.html This one looks more complicated - as near as I can tell, he's assuming that organizations never get stuck in poor solutions, so they can always pick the best of a sufficiently large crowd. Math and political correctness don't mix well.
newt0311 18y ago
Short summary: The best problem solvers will think of similar solutions and so may take longer to solve the problem than a random group because the random gorp will come up with more ways of solving the problem.

In my experience, diversity in temperament is essential for any team to function. If you put two people who are highly excitable together, they will keep running after new ideas and feeding of each others excitement. If you put two optimists together, they will tend to gamble and play too risky. If you put a team of idea men together, you tend to have designs and nobody doing the actual work. I believe every team should be made up of a bunch of slow and steady chaps, and a few creative people, with the creative people split into excitable and careful. The guys that one often assumes are slow usually have really good solutions, but when the team is too fast paced, those ideas get drowned out by the people who are loud and confident of their own ability. Soon, the quiet chaps stop talking, because they know nobody will listen to them. Diversity has nothing to do with ethnicity if you are faced with a programming task. If you were to solve a socio-cultural problem, then diversity in terms of ethnicity would probably be a good division criteria.

michael_nielsen 18y ago

The punchline, from the abstract: "We find that when selecting a problem-solving team from a diverse population of intelligent agents, a team of randomly selected agents outperforms a team comprised of the best-performing agents. This result relies on the intuition that, as the initial pool of problem solvers becomes large, the best-performing agents necessarily become similar in the space of problem solvers. Their relatively greater ability is more than offset by their lack of problem-solving diversity." It's not always true, of course, that a randomly selected group will outperform a group of top performers. But the result does suggest that it's worth thinking hard about the tradeoff between diversity and ability, and that sometimes the former is more important than the latter.

DanielBMarkham 18y ago

Very insightful. Excellent catch.

thras 18y ago

Unlikely. This is a theoretical paper, not based on any actual data. What the authors get wrong is assuming that there is no correlation between intelligence and original thinking. It is an unwarranted and frankly stupid assumption. The claim that diversity trumps ability is defeated in nearly every real world test. I notice that the author's advice, to hire a random sample of candidates for the position instead of the best, is not followed by any great companies that I can think of. In fact, it's 180 degrees away from the hiring practices of places like Google, etc.

maxklein 18y ago
How many 'original thinkers' do you need in a company made up of 50.000 people? I'm sure you're more intelligent than I am, but don't you think there are some problems I can solve better than you, based off my past experience? Diverse usually implies diverse in experience, and not in intelligence.
DanielBMarkham 18y ago
It's not an either-or choice. Best-case is to hire a highly-intelligent, intellectually diverse team. Bingo Presto, life is good.
- thras 18y ago
  Even then, I wonder whether maximizing intellectual diversity is really the best strategy. Teams with hefty philosophical disagreements can have trouble getting on the same page enough to accomplish goals. There may be an optimal level for intellectual diversity, but I wonder if it isn't closer to the minimum than the maximum for most tasks. It would be interesting to see real data on that; I have none.
  - DanielBMarkham 18y ago
    Yes. I think you have to add social skills into the mix. Ideally maximum intellectual diversity with maximum social (negotiation, facilitation, humility, leadership) abilities, but you'd have to scale back on intellectual diversity depending on the social ability of the group.

Eliezer 18y ago

Scott Page again? See here for a deconstruction of one of his previous, simpler arguments. http://www.overcomingbias.com/2007/04/the_error_of_cr.html http://www.overcomingbias.com/2007/04/the_error_of_cr.html This one looks more complicated - as near as I can tell, he's assuming that organizations never get stuck in poor solutions, so they can always pick the best of a sufficiently large crowd. Math and political correctness don't mix well.

newt0311 18y ago

Short summary: The best problem solvers will think of similar solutions and so may take longer to solve the problem than a random group because the random gorp will come up with more ways of solving the problem.