3 ms·
I would dispute that he (or I) have missed that point - he is simply giving an example as to why one should, as is always the case when undertaking statistical
by mrow84 13y ago
I would dispute that he (or I) have missed that point - he is simply giving an example as to why one should, as is always the case when undertaking statistical study, make a reasonable attempt to understand the population from which you are sampling.
The wisdom of crowds effect is not a 'strong result', and it can be hard to work out whether it will apply in any given situation, and so it is prudent to exercise some caution, and to teach people why said caution is warranted, which is the point of his little game.
The 'wisdom of crowds estimate' (take 5 samples and average) isn't a biased estimator, because averaged across all realisations it would produce a correct result. The bias creeps in mostly because of careless assumptions on the part of the samplers.
You said "that misses the whole point of why the "wisdom of crowds" actually works when it does" - I guess all he's saying is that it doesn't always work, which I presume you agree with, and he's giving an example of why. To flip your argument around: saying that the wisdom of crowds works when it works is simply tautological.
edit: just to clarify, the "crowd" here is the group of sweets, not the participants. The participants are multiple realisations of the estimator, at least that is my interpretation (because if not, then I agree with you - the participants aren't a "crowd" in that sense).
- ska 13y agore your edit: yes it was the latter interpretation that I had a problem with. The "group of sweets" == "crowd" doesn't really work for me, but I'll have to think about why.
- mrow84 13y agoYeah I only realised that you were interpreting it differently when I saw your reply to jimhefferon. If you do figure out what bothers you about the sweets==crowd interpretation then do please reply - statistics is so slippery that it takes a lot for me to feel fully confident with any particular idea.
- ska 13y agoOk, I think the simple version of this is: You are basically constructing a situation where the dominant bias is the same for all of the measurements (e.g. pairs of students). So you have a problem with your method of sampling being biased in a predictable way. So far, so good and this is an interesting lesson. But they are all using the same estimation method from this biased information, and they are all (by construction) creating the same bias, and that is very much not crowd-like behavior. Applying this as an example of the problem with "wisdom of crowds" seems to really miss the point to me, because you created a situation where they didn't behave like a crowd.
- mrow84 13y agoOn further reflection, I have to concede that it is not a particularly good example of the wisdom of crowds idea. Whilst I still think you can interpret his experiment as showing something about how and why WoC can go wrong, essentially along the lines I previously argued, it is not obvious, and there is some subtlety in the interpretation. Your original phrasing "the "wisdom of crowds" part of the title is a bit unfortunate" seems most appropriate; indeed the word "crowd" doesn't appear in the rest of the piece.