7 ms·

Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

111 points by todd8 12y ago

amathstudent 12y ago
As someone who knows it intimately, I would like to say that this kind of thing is endemic across the social science literature. What is frustrating is that only very few people seem to have the ability to understand why it is not sound.
- jev 12y ago
  Probably because the mathematicians don't seem to have the ability to explain why it is not sound without devolving into obscure symbols and jargon.
  - amathstudent 12y ago
    I think that's a very important point. Indeed, when you try to explain something simply to people, they often won't believe that it's really that simple, since there's all those symbols!
  - dxbydt 12y ago
    If this was html & written to the point instead of stuffed in a pdf & written up in this roundabout manner, it would be on top of HN with 100+ votes :) I read the whole pdf but here's the TL:DR; anyways - Randomization improves algorithms. This is so obvious to CS folks its taught in basic cs101 - the ones i'm familiar with are where you pick a random pivot element for quicksort, or say the one where you draw a circle inside a square & pick n random points inside the square, four times the number of points inside the circle will equal pi as n becomes large, or you resort to the miller rabin test for primality when you are doing those rsa calculator type problems where you pick keys for encryption, which will randomly sample some number of possible witnesses and call the number prime if none turn out to be witnesses, or using monte-carlo methods to compute integrals for functions for which no closed-form formula exists etc....there's like tons of examples where you introduce a little bit of randomness & it'll speed up your algorithm. So this Page+Hong wrote a paper where they want you to randomly hire employees ( who they call problem-solvers or agents ), because diversity trumps ability ?! So if you have 1000 applicants, instead of hiring based on some metric like ivy-league/test-scores/IQ/github/whatever, you just hire randomly, because the randomness introduces diversity which trumps ability ?!! To prove this nonsensical point, they introduce an artificial math problem where agents proceeding randomly obtain the right answer, & not doing so gets you stuck. Ergo, diversity > ability. Sheesh.
    - yummyfajitas 12y ago
      This is in Notices of the AMS, which is about as mainstream (together with SIAM Review) as mathematical publications get. Randomization improves algorithms. This is a contentious claim. Random choices are actually very rarely the best - they are often good enough and versatile, but they are rarely the best. For example, consider Monte Carlo integration. You get O(N^{-1/2}) convergence. If you use a deterministic set of points explicitly designed to have low discrepancy (aka "Quasi-Monte Carlo"), you can get O(N^{-1 + logarithmic stuff}) convergence. http://www.chrisstucchio.com/blog/2014/adversarial_bandit_is_not_statistics_problem.html http://www.chrisstucchio.com/blog/2014/adversarial_bandit_is... Eliezer Yudkowsky also wrote a great critique of this issue, though I can't find it right now.
      - zwegner 12y ago
        I believe the article you're referring to is: http://lesswrong.com/lw/vp/worse_than_random/ http://lesswrong.com/lw/vp/worse_than_random/ Choice quote: > As a general principle, on any problem for which you know that a particular unrandomized algorithm is unusually stupid - so that a randomized algorithm seems wiser - you should be able to use the same knowledge to produce a superior derandomized algorithm. An interesting counterexample to this, an example which to me illustrates a very powerful aspect of randomness, is the monte-carlo revolution in computer Go (AI for the ancient Asian board game). For years, computer progress stagnated, as the techniques were mostly focused on encoding human knowledge into code. While most of this human knowledge is generally correct, it introduces significant bias in the way positions are evaluated. The way these rules-of-thumb interact is very hard to predict, and tree search algorithms are quite good at finding positions that are incorrectly evaluated. Because of this, the worst-case behavior of an evaluation function is much more important than it's best-case or average behavior. Computers started making lots of progress when a new technique was used: rather than try to evaluate positions by a large set of heuristics, they were evaluated by playing random games. Go positions are quite hard to evaluate from simple rules of thumb, and this random game approach gives a much more balanced, long-term view of positions. And most importantly, there is much less bias. Obviously, an entirely random game, with a uniform distribution over possible moves, is easy to improve upon. But computer go programmers noticed an interesting phenomenon: while certain types of knowledge incorporated into the random move distribution (to make it "more intelligent", as judged by a human) were helpful, others were not (even after taking into account the computational cost of adding the knowledge), and it wasn't always clear why. The same observation about heuristic evaluation noted above applied: having a balanced distribution of move choices, with a reasonable probabilistic lower bound of effectiveness, is more important than making an intelligent choice that is usually correct, but has unpredictable, extreme worst-case performance. So we see that randomness does have an important property: it avoids the downside of "knowledge" that generally seems correct but can go horribly wrong in unexpected ways. I don't know of a good writeup of this phenomenon. My understanding of it is mostly assembled from following informal discussions on the computer-go mailing list for several years. In a quick search through my gmail archives I can't find much on the subject, but here's an interesting post about related topics in the computer chess world (that incidentally doesn't talk about randomness, but illustrates well the benefits of avoiding bias): http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=135133&t=15504 http://www.talkchess.com/forum/viewtopic.php?topic_view=thre...
  - revelation 12y ago
    Have you read the linked paper and the one its discussing? I find the critique vastly more readable.
- kitsune_ 12y ago
  Because I didn't have any money after high school I decided to get a degree where I could work part time. I had been programming for quite some years but had an interest in media and politics. So I started to study media sciences at the largest university here in Switzerland. After a month I realized that I had made a huge mistake. There was a class called 'statistics in social sciences' and the assistants had trouble with basic stuff like calculating the mean, median, avg etc. They even celebrated their mathematical illiteracy to a certain degree. On top of that, much of the writing in the standard text books could be reduced to meaningless tautologies once you deconstructed the complex phrases a bit. I dropped out very quickly after that. The whole experience left a very sour taste in my mouth.
nostromo 12y ago
I was shocked to read the original article. I can't believe that such overarching conclusions about human nature were drawn from a little toy box computational experiment. You might as well publish a psychology paper after a weekend playing The Sims. http://vserver1.cscs.lsa.umich.edu/~spage/pnas.pdf http://vserver1.cscs.lsa.umich.edu/~spage/pnas.pdf
- kenjackson 12y ago
  It also seems to defy common sense. How many top ACM programming teams randomly pick programmers from the college? How many great bball teams are randomly picked from players? There are few activities, where excellence is easily measured, where randomly picking participants results in superior results.
  - johnbender 12y ago
    It's important to realize that common sense is never a replacement for a proof or emperical result. That is, common sense is not sufficient to refute the claims made by the original paper. The power of science is that it often defies reasoning and in so doing provides new insights into how things work.
    - tbrownaw 12y ago
      But a clearly absurd or nonsensical result should be enough reason to double-check that the proof and results are actually valid. Common sense may have a somewhat low weight for evaluating how probable some scientific result is, but it doesn't have zero weight. And neither does the possibility of experimental error.
      - deleted 12y ago
        [deleted]
      - waps 12y ago
        From the original paper : Lu Hong†‡§ and Scott E. Page¶ †Michigan Business School and ¶Complex Systems, University of Michigan, Ann Arbor, MI 48109-1234; and ‡Department of Finance, Loyola University, Chicago, IL 60611 Now that "Business school" part, that's a good reason to recheck the conclusions. Heh, I'd like to pretend that a rebuttal to business school ideas will actually change them, but we all know how this will play out. The idea of diversity is imposed from above, and no-one will care about this. Politics is the origin of the diversity idea, nothing else.
HelloMcFly 12y ago
If anyone is interested in better research on diversity and team outcomes, I recommend looking into research on "faultlines" in teams. It is an interesting way of operationalizing diversity and predicting its effects. Link to abstract: http://amr.aom.org/content/23/2/325.short http://amr.aom.org/content/23/2/325.short I can't find a direct link to the PDF at the moment. That's the original article. There's been a lot of research since testing the theory, typically supporting it (though I'm not 100% up to date on this topic).
bayesianhorse 12y ago
In finance it has long been known that diversity in a portfolio can often trump the performance/ability in a particular stock. Turns out, you can see portfolio theory popping up outside of financial markets, not only in assembling a team, but also advertising campaigns, ant colonies and bacterial colonies. And yes, I know that the linked article debunks a paper that abuses math... It's just that portfolio theory is at least an analogy to understand why diversity can trump ability. Within reason.
- Guthur 12y ago
  In my opinion the reason "portfolio theory" works in the cases that you mentioned is that it most perform well across a time series within a highly dynamic environment. But if one is allow to pick an optimal team for each discrete problem then it will surely outperform. In my mind diversity will help when you have little to no a priori knowledge and then for can not predict what attributes you will need.
  - bayesianhorse 12y ago
    Have you never seen a software development project which ran into unanticipated problems? What about "knowledge work" is not a highly dynamic environment?
- ohsnap 12y ago
  Portfolio theory does not work here. Totally different concepts. MPT is about minimizing risk - not trumping the performance of a stock.
  - bayesianhorse 12y ago
    MPT is not about "minimizing the risk". It's about minimizing risk for a desired reward. In an abstract way of speaking, selecting team members is very similar.
- roguecoder 12y ago
  Also the empirical studies from MIT about group problem solving. They have observed that diversity contributes more than the abilities of the individuals. It's not that the idea that diversity trumps ability in an unpredictable situation is wrong: it's that the original mathematical argument doesn't demonstrate that.
deleted 12y ago
[deleted]
cjdrake 12y ago
Richard Feynman has some relevant thoughts on this subject: https://www.youtube.com/watch?v=IaO69CF5mbY https://www.youtube.com/watch?v=IaO69CF5mbY
nether 12y ago
> Page’s work on diversity has been cited by NASA, the US Geological Survey, and Lawrence Berkeley Labs, among many others Oops.
nether 12y ago
One of the authors, Scott Page, majored in math at the University of Michigan. He also teaches the course "Model Thinking" on Coursera: https://www.coursera.org/course/modelthinking https://www.coursera.org/course/modelthinking. Perhaps one to avoid...
- dxbydt 12y ago
  Hey that's a great course. Pls do not avoid. Highly recommend signing up. The reason he wrote this famous paper & the book that evolved from that paper, in his own words - http://vserver1.cscs.lsa.umich.edu/~spage/thedifference_interview.html http://vserver1.cscs.lsa.umich.edu/~spage/thedifference_inte... He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together" Not too sure about that. At all.Especially not in math, the subject he majored in.
  - eropple 12y ago
    > He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together" > Not too sure about that. Why would that not be a thing? I mean, I am not the smartest person on my team. The list of things I know very little about is long. But I occasionally ask questions that are really obvious to me that reorient the discussion because people didn't think of them--problems that were Too Obvious To See up close, you know? And, similarly, when working in stuff that I do know a lot about, I find myself ignoring things that are to me so obvious and basic that my brain just goes right by them. Innovation isn't like Civilization, you aren't just generating lightbulbs. Diversity of thought process leads to some inefficiencies, but it helps you reach new global maxima.
    - jrapdx3 12y ago
      I'm not so sure diverse (divergent?) approaches to a problem are always better than one leader's effort. For one thing, stand-out genius or talent shows up randomly in populations, it's not really predictable when or where it will occur. But on rare occasion when a natural leader does emerge, suppressing diversity (of problem-solving approaches) is likely the better strategy. The true maxima of human effort have generally followed this pattern, born of singularity and not diversity. Once an innovation is widely enough known, it attracts a diverse range of followers who improve the idea and nurture its maturation. Diversity is useful for the "aftercare" of innovation, but not its creation. The other major point is that value of diversity (however defined) vs. uniformity depends on context. In the context of one's home, there may be many ways to arrange furniture in a room, and with few exceptions, one way is as valid as another. By contrast, in a shared household over time diverse opinions are likely to converge to an arrangement using the space optimally. Diversity probably leads to a better workflow solution vs. one person's choices. OTOH there is not a great diversity of "valid" ways to remove an inflamed appendix, that is, regardless of other considerations, a qualified surgeon is required and procedural diversity is constrained by the anatomical realities. This situation demands uniformity not diversity. Finally, "diversity" has an indeterminate number of definitions, and applicability of the term is entirely dependent on context. On second thought, that would seem to drain it of specificity or meaning, on that ground, I should have a policy of using "diversity" carefully and sparingly.
deleted 12y ago
[deleted]
Russell91 12y ago
Am I missing something or is the author of this pdf's proof equally flawed. They assume each function is idempotent and injective ... meaning that it is the identity function. And the proof doesn't follow. A much more natural way to have "fixed" the original proof would be to require that V: X -> R be injective.
- 4bpp 12y ago
  Assuming by function you refer to the "agents'" \Phi, where do they make the assumption that it is injective? I see nothing to the effect in the text, and the functions in the counterexample on page 6 are not injective.
  - Russell91 12y ago
    Oh, oops. I was reading their statement: This interpretation is incorrect without the additional hypothesis that V (x) is a one-to-one function. as - all of the phi functions must be one-to-one. Looks like they made the same assumption that I would have. This was bothering me though so thanks for your response.
4bpp 12y ago
The response paper seems to harp a lot on how the circumstance that typically, N_1 and N >> k, precludes any realistic applicability of the original theorem -- but it admits a fairly self-consistent and altogether much more sinister interpretation if you call the different \Phi "groups" (or "races" for maximum creepiness). The assumptions then say something to the effect of "every group's approach to problem solving produces fundamentally predictable results, and no group can be the best at solving each type of problem", which is a meme that in some form has been floating around in diversity activism circles for a long time. The "mathematical theorem" then simply captures the triviality that under this sort of worldview, you want your team to contain Zorblaxians (sorry, SMBC) because there are some Zorblaxian problems that Zorblaxians have a natural affinity towards, and nobody without such an affinity could possibly make progress on. Since this is not stated explicitly, the statement becomes invokable even in settings in which otherwise a large number of people would raise eyebrows at the smell of exoticist quackery that the idea of "different ways of knowing" exudes.
- QuantumChaos 12y ago
  Your post captures the problem much better than the linked article. I would summarize the problem as being that the result is tautological, but the gloss of mathematics gives it the appearance of not being tautological. If people said "diverse teams are better because people from different backgrounds are good at solving different kids of problems" then that would be accurate, but leave room for debate about whether then assumption is true. If they say "diverse teams are mathematically proven to be better", this would be inaccurate, but give great ammunition to argue that science supports progressive views.
im3w1l 12y ago
Has there been any attempt at quantifying the "wishful thinking bias" in social science?
- amathstudent 12y ago
  Here's something you might find interesting: http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/698.html http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/698.htm... There have are also loads of replication projects, some currently ongoing, of both experimental and statistical papers, with generally dismal results.
rlvesco7 12y ago
There is one positive aspect to all this that I would like to point out. Most social science does not use math, so it makes it hard to argue against ill-defined concepts and ideas. So the fact that these mistakes were found is actually a positive thing and shows one reason why math is useful. That's not to say math cannot be used to obscure arguments and assumptions, it can, but words can do that too.
pervycreeper 12y ago
That was surprisingly entertaining. Any recommendations for other academic papers in that vein?

111 points by todd8 12y ago

As someone who knows it intimately, I would like to say that this kind of thing is endemic across the social science literature. What is frustrating is that only very few people seem to have the ability to understand why it is not sound.

jev 12y ago
Probably because the mathematicians don't seem to have the ability to explain why it is not sound without devolving into obscure symbols and jargon.
- amathstudent 12y ago
  I think that's a very important point. Indeed, when you try to explain something simply to people, they often won't believe that it's really that simple, since there's all those symbols!
- dxbydt 12y ago
  If this was html & written to the point instead of stuffed in a pdf & written up in this roundabout manner, it would be on top of HN with 100+ votes :) I read the whole pdf but here's the TL:DR; anyways - Randomization improves algorithms. This is so obvious to CS folks its taught in basic cs101 - the ones i'm familiar with are where you pick a random pivot element for quicksort, or say the one where you draw a circle inside a square & pick n random points inside the square, four times the number of points inside the circle will equal pi as n becomes large, or you resort to the miller rabin test for primality when you are doing those rsa calculator type problems where you pick keys for encryption, which will randomly sample some number of possible witnesses and call the number prime if none turn out to be witnesses, or using monte-carlo methods to compute integrals for functions for which no closed-form formula exists etc....there's like tons of examples where you introduce a little bit of randomness & it'll speed up your algorithm. So this Page+Hong wrote a paper where they want you to randomly hire employees ( who they call problem-solvers or agents ), because diversity trumps ability ?! So if you have 1000 applicants, instead of hiring based on some metric like ivy-league/test-scores/IQ/github/whatever, you just hire randomly, because the randomness introduces diversity which trumps ability ?!! To prove this nonsensical point, they introduce an artificial math problem where agents proceeding randomly obtain the right answer, & not doing so gets you stuck. Ergo, diversity > ability. Sheesh.
  - yummyfajitas 12y ago
    This is in Notices of the AMS, which is about as mainstream (together with SIAM Review) as mathematical publications get. Randomization improves algorithms. This is a contentious claim. Random choices are actually very rarely the best - they are often good enough and versatile, but they are rarely the best. For example, consider Monte Carlo integration. You get O(N^{-1/2}) convergence. If you use a deterministic set of points explicitly designed to have low discrepancy (aka "Quasi-Monte Carlo"), you can get O(N^{-1 + logarithmic stuff}) convergence. http://www.chrisstucchio.com/blog/2014/adversarial_bandit_is_not_statistics_problem.html http://www.chrisstucchio.com/blog/2014/adversarial_bandit_is... Eliezer Yudkowsky also wrote a great critique of this issue, though I can't find it right now.
    - zwegner 12y ago
      I believe the article you're referring to is: http://lesswrong.com/lw/vp/worse_than_random/ http://lesswrong.com/lw/vp/worse_than_random/ Choice quote: > As a general principle, on any problem for which you know that a particular unrandomized algorithm is unusually stupid - so that a randomized algorithm seems wiser - you should be able to use the same knowledge to produce a superior derandomized algorithm. An interesting counterexample to this, an example which to me illustrates a very powerful aspect of randomness, is the monte-carlo revolution in computer Go (AI for the ancient Asian board game). For years, computer progress stagnated, as the techniques were mostly focused on encoding human knowledge into code. While most of this human knowledge is generally correct, it introduces significant bias in the way positions are evaluated. The way these rules-of-thumb interact is very hard to predict, and tree search algorithms are quite good at finding positions that are incorrectly evaluated. Because of this, the worst-case behavior of an evaluation function is much more important than it's best-case or average behavior. Computers started making lots of progress when a new technique was used: rather than try to evaluate positions by a large set of heuristics, they were evaluated by playing random games. Go positions are quite hard to evaluate from simple rules of thumb, and this random game approach gives a much more balanced, long-term view of positions. And most importantly, there is much less bias. Obviously, an entirely random game, with a uniform distribution over possible moves, is easy to improve upon. But computer go programmers noticed an interesting phenomenon: while certain types of knowledge incorporated into the random move distribution (to make it "more intelligent", as judged by a human) were helpful, others were not (even after taking into account the computational cost of adding the knowledge), and it wasn't always clear why. The same observation about heuristic evaluation noted above applied: having a balanced distribution of move choices, with a reasonable probabilistic lower bound of effectiveness, is more important than making an intelligent choice that is usually correct, but has unpredictable, extreme worst-case performance. So we see that randomness does have an important property: it avoids the downside of "knowledge" that generally seems correct but can go horribly wrong in unexpected ways. I don't know of a good writeup of this phenomenon. My understanding of it is mostly assembled from following informal discussions on the computer-go mailing list for several years. In a quick search through my gmail archives I can't find much on the subject, but here's an interesting post about related topics in the computer chess world (that incidentally doesn't talk about randomness, but illustrates well the benefits of avoiding bias): http://www.talkchess.com/forum/viewtopic.php?topic_view=threads&p=135133&t=15504 http://www.talkchess.com/forum/viewtopic.php?topic_view=thre...
- revelation 12y ago
  Have you read the linked paper and the one its discussing? I find the critique vastly more readable.
kitsune_ 12y ago
Because I didn't have any money after high school I decided to get a degree where I could work part time. I had been programming for quite some years but had an interest in media and politics. So I started to study media sciences at the largest university here in Switzerland. After a month I realized that I had made a huge mistake. There was a class called 'statistics in social sciences' and the assistants had trouble with basic stuff like calculating the mean, median, avg etc. They even celebrated their mathematical illiteracy to a certain degree. On top of that, much of the writing in the standard text books could be reduced to meaningless tautologies once you deconstructed the complex phrases a bit. I dropped out very quickly after that. The whole experience left a very sour taste in my mouth.

nostromo 12y ago

I was shocked to read the original article. I can't believe that such overarching conclusions about human nature were drawn from a little toy box computational experiment. You might as well publish a psychology paper after a weekend playing The Sims. http://vserver1.cscs.lsa.umich.edu/~spage/pnas.pdf http://vserver1.cscs.lsa.umich.edu/~spage/pnas.pdf

kenjackson 12y ago
It also seems to defy common sense. How many top ACM programming teams randomly pick programmers from the college? How many great bball teams are randomly picked from players? There are few activities, where excellence is easily measured, where randomly picking participants results in superior results.
- johnbender 12y ago
  It's important to realize that common sense is never a replacement for a proof or emperical result. That is, common sense is not sufficient to refute the claims made by the original paper. The power of science is that it often defies reasoning and in so doing provides new insights into how things work.
  - tbrownaw 12y ago
    But a clearly absurd or nonsensical result should be enough reason to double-check that the proof and results are actually valid. Common sense may have a somewhat low weight for evaluating how probable some scientific result is, but it doesn't have zero weight. And neither does the possibility of experimental error.
    - deleted 12y ago
      [deleted]
    - waps 12y ago
      From the original paper : Lu Hong†‡§ and Scott E. Page¶ †Michigan Business School and ¶Complex Systems, University of Michigan, Ann Arbor, MI 48109-1234; and ‡Department of Finance, Loyola University, Chicago, IL 60611 Now that "Business school" part, that's a good reason to recheck the conclusions. Heh, I'd like to pretend that a rebuttal to business school ideas will actually change them, but we all know how this will play out. The idea of diversity is imposed from above, and no-one will care about this. Politics is the origin of the diversity idea, nothing else.

HelloMcFly 12y ago

If anyone is interested in better research on diversity and team outcomes, I recommend looking into research on "faultlines" in teams. It is an interesting way of operationalizing diversity and predicting its effects. Link to abstract: http://amr.aom.org/content/23/2/325.short http://amr.aom.org/content/23/2/325.short I can't find a direct link to the PDF at the moment. That's the original article. There's been a lot of research since testing the theory, typically supporting it (though I'm not 100% up to date on this topic).

bayesianhorse 12y ago

In finance it has long been known that diversity in a portfolio can often trump the performance/ability in a particular stock. Turns out, you can see portfolio theory popping up outside of financial markets, not only in assembling a team, but also advertising campaigns, ant colonies and bacterial colonies. And yes, I know that the linked article debunks a paper that abuses math... It's just that portfolio theory is at least an analogy to understand why diversity can trump ability. Within reason.

Guthur 12y ago
In my opinion the reason "portfolio theory" works in the cases that you mentioned is that it most perform well across a time series within a highly dynamic environment. But if one is allow to pick an optimal team for each discrete problem then it will surely outperform. In my mind diversity will help when you have little to no a priori knowledge and then for can not predict what attributes you will need.
- bayesianhorse 12y ago
  Have you never seen a software development project which ran into unanticipated problems? What about "knowledge work" is not a highly dynamic environment?
ohsnap 12y ago
Portfolio theory does not work here. Totally different concepts. MPT is about minimizing risk - not trumping the performance of a stock.
- bayesianhorse 12y ago
  MPT is not about "minimizing the risk". It's about minimizing risk for a desired reward. In an abstract way of speaking, selecting team members is very similar.
roguecoder 12y ago
Also the empirical studies from MIT about group problem solving. They have observed that diversity contributes more than the abilities of the individuals. It's not that the idea that diversity trumps ability in an unpredictable situation is wrong: it's that the original mathematical argument doesn't demonstrate that.

deleted 12y ago

[deleted]

cjdrake 12y ago

Richard Feynman has some relevant thoughts on this subject: https://www.youtube.com/watch?v=IaO69CF5mbY https://www.youtube.com/watch?v=IaO69CF5mbY

nether 12y ago

> Page’s work on diversity has been cited by NASA, the US Geological Survey, and Lawrence Berkeley Labs, among many others Oops.

nether 12y ago

One of the authors, Scott Page, majored in math at the University of Michigan. He also teaches the course "Model Thinking" on Coursera: https://www.coursera.org/course/modelthinking https://www.coursera.org/course/modelthinking. Perhaps one to avoid...

dxbydt 12y ago
Hey that's a great course. Pls do not avoid. Highly recommend signing up. The reason he wrote this famous paper & the book that evolved from that paper, in his own words - http://vserver1.cscs.lsa.umich.edu/~spage/thedifference_interview.html http://vserver1.cscs.lsa.umich.edu/~spage/thedifference_inte... He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together" Not too sure about that. At all.Especially not in math, the subject he majored in.
- eropple 12y ago
  > He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together" > Not too sure about that. Why would that not be a thing? I mean, I am not the smartest person on my team. The list of things I know very little about is long. But I occasionally ask questions that are really obvious to me that reorient the discussion because people didn't think of them--problems that were Too Obvious To See up close, you know? And, similarly, when working in stuff that I do know a lot about, I find myself ignoring things that are to me so obvious and basic that my brain just goes right by them. Innovation isn't like Civilization, you aren't just generating lightbulbs. Diversity of thought process leads to some inefficiencies, but it helps you reach new global maxima.
  - jrapdx3 12y ago
    I'm not so sure diverse (divergent?) approaches to a problem are always better than one leader's effort. For one thing, stand-out genius or talent shows up randomly in populations, it's not really predictable when or where it will occur. But on rare occasion when a natural leader does emerge, suppressing diversity (of problem-solving approaches) is likely the better strategy. The true maxima of human effort have generally followed this pattern, born of singularity and not diversity. Once an innovation is widely enough known, it attracts a diverse range of followers who improve the idea and nurture its maturation. Diversity is useful for the "aftercare" of innovation, but not its creation. The other major point is that value of diversity (however defined) vs. uniformity depends on context. In the context of one's home, there may be many ways to arrange furniture in a room, and with few exceptions, one way is as valid as another. By contrast, in a shared household over time diverse opinions are likely to converge to an arrangement using the space optimally. Diversity probably leads to a better workflow solution vs. one person's choices. OTOH there is not a great diversity of "valid" ways to remove an inflamed appendix, that is, regardless of other considerations, a qualified surgeon is required and procedural diversity is constrained by the anatomical realities. This situation demands uniformity not diversity. Finally, "diversity" has an indeterminate number of definitions, and applicability of the term is entirely dependent on context. On second thought, that would seem to drain it of specificity or meaning, on that ground, I should have a policy of using "diversity" carefully and sparingly.

deleted 12y ago

[deleted]

Russell91 12y ago

Am I missing something or is the author of this pdf's proof equally flawed. They assume each function is idempotent and injective ... meaning that it is the identity function. And the proof doesn't follow. A much more natural way to have "fixed" the original proof would be to require that V: X -> R be injective.

4bpp 12y ago
Assuming by function you refer to the "agents'" \Phi, where do they make the assumption that it is injective? I see nothing to the effect in the text, and the functions in the counterexample on page 6 are not injective.
- Russell91 12y ago
  Oh, oops. I was reading their statement: This interpretation is incorrect without the additional hypothesis that V (x) is a one-to-one function. as - all of the phi functions must be one-to-one. Looks like they made the same assumption that I would have. This was bothering me though so thanks for your response.

4bpp 12y ago

The response paper seems to harp a lot on how the circumstance that typically, N_1 and N >> k, precludes any realistic applicability of the original theorem -- but it admits a fairly self-consistent and altogether much more sinister interpretation if you call the different \Phi "groups" (or "races" for maximum creepiness). The assumptions then say something to the effect of "every group's approach to problem solving produces fundamentally predictable results, and no group can be the best at solving each type of problem", which is a meme that in some form has been floating around in diversity activism circles for a long time. The "mathematical theorem" then simply captures the triviality that under this sort of worldview, you want your team to contain Zorblaxians (sorry, SMBC) because there are some Zorblaxian problems that Zorblaxians have a natural affinity towards, and nobody without such an affinity could possibly make progress on. Since this is not stated explicitly, the statement becomes invokable even in settings in which otherwise a large number of people would raise eyebrows at the smell of exoticist quackery that the idea of "different ways of knowing" exudes.

QuantumChaos 12y ago
Your post captures the problem much better than the linked article. I would summarize the problem as being that the result is tautological, but the gloss of mathematics gives it the appearance of not being tautological. If people said "diverse teams are better because people from different backgrounds are good at solving different kids of problems" then that would be accurate, but leave room for debate about whether then assumption is true. If they say "diverse teams are mathematically proven to be better", this would be inaccurate, but give great ammunition to argue that science supports progressive views.

im3w1l 12y ago

Has there been any attempt at quantifying the "wishful thinking bias" in social science?

amathstudent 12y ago
Here's something you might find interesting: http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/698.html http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/698.htm... There have are also loads of replication projects, some currently ongoing, of both experimental and statistical papers, with generally dismal results.

rlvesco7 12y ago

There is one positive aspect to all this that I would like to point out. Most social science does not use math, so it makes it hard to argue against ill-defined concepts and ideas. So the fact that these mistakes were found is actually a positive thing and shows one reason why math is useful. That's not to say math cannot be used to obscure arguments and assumptions, it can, but words can do that too.

pervycreeper 12y ago

That was surprisingly entertaining. Any recommendations for other academic papers in that vein?