4 ms·

I think the point is that you are taking the weights of the candies as proxies for the guesses of individuals - the (sum * 20) procedure is effectively an avera

by mrow84 13y ago

I think the point is that you are taking the weights of the candies as proxies for the guesses of individuals - the (sum * 20) procedure is effectively an average-based estimate, like with the 'traditional' wisdom of crowds type approach. He is then simply showing that for certain populations such approaches won't necessarily give good expected results, and so one should always exercise caution - whether the population is sweets or people's guesses at the weight of a cow is essentially immaterial.

ska 13y ago
But that misses the whole point of why the "wisdom of crowds" actually works when it does. To wit; the individual biases and estimation errors can tend to cancel out. This is exactly the opposite of the demonstration, which is a systemic, biased estimator.
- jimhefferon 13y ago
  Perhaps you missed that the title says unwisdom? That is, not-the wisdom.
  - ska 13y ago
    No, I didn't miss that. It the lack of "crowd" that is the problem. There is no crowd, really, there are a bunch of people performing a sampling algorithm. But crowds don't work that way. This is a perfectly good demonstration of bias in statistical estimators. It just doesn't have anything to do with crowds, their wisdom, or their unwisdom.
- mrow84 13y ago
  I would dispute that he (or I) have missed that point - he is simply giving an example as to why one should, as is always the case when undertaking statistical study, make a reasonable attempt to understand the population from which you are sampling. The wisdom of crowds effect is not a 'strong result', and it can be hard to work out whether it will apply in any given situation, and so it is prudent to exercise some caution, and to teach people why said caution is warranted, which is the point of his little game. The 'wisdom of crowds estimate' (take 5 samples and average) isn't a biased estimator, because averaged across all realisations it would produce a correct result. The bias creeps in mostly because of careless assumptions on the part of the samplers. You said "that misses the whole point of why the "wisdom of crowds" actually works when it does" - I guess all he's saying is that it doesn't always work, which I presume you agree with, and he's giving an example of why. To flip your argument around: saying that the wisdom of crowds works when it works is simply tautological. edit: just to clarify, the "crowd" here is the group of sweets, not the participants. The participants are multiple realisations of the estimator, at least that is my interpretation (because if not, then I agree with you - the participants aren't a "crowd" in that sense).
  - ska 13y ago
    re your edit: yes it was the latter interpretation that I had a problem with. The "group of sweets" == "crowd" doesn't really work for me, but I'll have to think about why.
    - mrow84 13y ago
      Yeah I only realised that you were interpreting it differently when I saw your reply to jimhefferon. If you do figure out what bothers you about the sweets==crowd interpretation then do please reply - statistics is so slippery that it takes a lot for me to feel fully confident with any particular idea.
      - ska 13y ago
        Ok, I think the simple version of this is: You are basically constructing a situation where the dominant bias is the same for all of the measurements (e.g. pairs of students). So you have a problem with your method of sampling being biased in a predictable way. So far, so good and this is an interesting lesson. But they are all using the same estimation method from this biased information, and they are all (by construction) creating the same bias, and that is very much not crowd-like behavior. Applying this as an example of the problem with "wisdom of crowds" seems to really miss the point to me, because you created a situation where they didn't behave like a crowd.
        mrow84 13y ago
        On further reflection, I have to concede that it is not a particularly good example of the wisdom of crowds idea. Whilst I still think you can interpret his experiment as showing something about how and why WoC can go wrong, essentially along the lines I previously argued, it is not obvious, and there is some subtlety in the interpretation. Your original phrasing "the "wisdom of crowds" part of the title is a bit unfortunate" seems most appropriate; indeed the word "crowd" doesn't appear in the rest of the piece.

by mrow84 13y ago

ska 13y ago

But that misses the whole point of why the "wisdom of crowds" actually works when it does. To wit; the individual biases and estimation errors can tend to cancel out. This is exactly the opposite of the demonstration, which is a systemic, biased estimator.

jimhefferon 13y ago
Perhaps you missed that the title says unwisdom? That is, not-the wisdom.
- ska 13y ago
  No, I didn't miss that. It the lack of "crowd" that is the problem. There is no crowd, really, there are a bunch of people performing a sampling algorithm. But crowds don't work that way. This is a perfectly good demonstration of bias in statistical estimators. It just doesn't have anything to do with crowds, their wisdom, or their unwisdom.
mrow84 13y ago
I would dispute that he (or I) have missed that point - he is simply giving an example as to why one should, as is always the case when undertaking statistical study, make a reasonable attempt to understand the population from which you are sampling. The wisdom of crowds effect is not a 'strong result', and it can be hard to work out whether it will apply in any given situation, and so it is prudent to exercise some caution, and to teach people why said caution is warranted, which is the point of his little game. The 'wisdom of crowds estimate' (take 5 samples and average) isn't a biased estimator, because averaged across all realisations it would produce a correct result. The bias creeps in mostly because of careless assumptions on the part of the samplers. You said "that misses the whole point of why the "wisdom of crowds" actually works when it does" - I guess all he's saying is that it doesn't always work, which I presume you agree with, and he's giving an example of why. To flip your argument around: saying that the wisdom of crowds works when it works is simply tautological. edit: just to clarify, the "crowd" here is the group of sweets, not the participants. The participants are multiple realisations of the estimator, at least that is my interpretation (because if not, then I agree with you - the participants aren't a "crowd" in that sense).
- ska 13y ago
  re your edit: yes it was the latter interpretation that I had a problem with. The "group of sweets" == "crowd" doesn't really work for me, but I'll have to think about why.
  - mrow84 13y ago
    Yeah I only realised that you were interpreting it differently when I saw your reply to jimhefferon. If you do figure out what bothers you about the sweets==crowd interpretation then do please reply - statistics is so slippery that it takes a lot for me to feel fully confident with any particular idea.
    - ska 13y ago
      Ok, I think the simple version of this is: You are basically constructing a situation where the dominant bias is the same for all of the measurements (e.g. pairs of students). So you have a problem with your method of sampling being biased in a predictable way. So far, so good and this is an interesting lesson. But they are all using the same estimation method from this biased information, and they are all (by construction) creating the same bias, and that is very much not crowd-like behavior. Applying this as an example of the problem with "wisdom of crowds" seems to really miss the point to me, because you created a situation where they didn't behave like a crowd.
      - mrow84 13y ago
        On further reflection, I have to concede that it is not a particularly good example of the wisdom of crowds idea. Whilst I still think you can interpret his experiment as showing something about how and why WoC can go wrong, essentially along the lines I previously argued, it is not obvious, and there is some subtlety in the interpretation. Your original phrasing "the "wisdom of crowds" part of the title is a bit unfortunate" seems most appropriate; indeed the word "crowd" doesn't appear in the rest of the piece.