5 ms·
The "wisdom of crowds" part of the title is a bit unfortunate. While there can be systemic problems with that type of approach, this does not really demonstrat
by ska 13y ago
The "wisdom of crowds" part of the title is a bit unfortunate. While there can be systemic problems with that type of approach, this does not really demonstrate them.
What is demonstrated is when you give the students an algorithm for a biased estimator, the estimate they get is biased. This is good; empirical demonstration is useful... but it isn't the wisdom/unwisdom of crowds, really.
edit: good responses! Unfortunately I don't have enough time right now to properly clarify how/why I'm looking at it this way.
- mrow84 13y agoI think the point is that you are taking the weights of the candies as proxies for the guesses of individuals - the (sum * 20) procedure is effectively an average-based estimate, like with the 'traditional' wisdom of crowds type approach. He is then simply showing that for certain populations such approaches won't necessarily give good expected results, and so one should always exercise caution - whether the population is sweets or people's guesses at the weight of a cow is essentially immaterial.
- ska 13y agoBut that misses the whole point of why the "wisdom of crowds" actually works when it does. To wit; the individual biases and estimation errors can tend to cancel out. This is exactly the opposite of the demonstration, which is a systemic, biased estimator.
- jimhefferon 13y agoPerhaps you missed that the title says unwisdom? That is, not-the wisdom.
- ska 13y agoNo, I didn't miss that. It the lack of "crowd" that is the problem. There is no crowd, really, there are a bunch of people performing a sampling algorithm. But crowds don't work that way. This is a perfectly good demonstration of bias in statistical estimators. It just doesn't have anything to do with crowds, their wisdom, or their unwisdom.
- mrow84 13y agoI would dispute that he (or I) have missed that point - he is simply giving an example as to why one should, as is always the case when undertaking statistical study, make a reasonable attempt to understand the population from which you are sampling. The wisdom of crowds effect is not a 'strong result', and it can be hard to work out whether it will apply in any given situation, and so it is prudent to exercise some caution, and to teach people why said caution is warranted, which is the point of his little game. The 'wisdom of crowds estimate' (take 5 samples and average) isn't a biased estimator, because averaged across all realisations it would produce a correct result. The bias creeps in mostly because of careless assumptions on the part of the samplers. You said "that misses the whole point of why the "wisdom of crowds" actually works when it does" - I guess all he's saying is that it doesn't always work, which I presume you agree with, and he's giving an example of why. To flip your argument around: saying that the wisdom of crowds works when it works is simply tautological. edit: just to clarify, the "crowd" here is the group of sweets, not the participants. The participants are multiple realisations of the estimator, at least that is my interpretation (because if not, then I agree with you - the participants aren't a "crowd" in that sense).
- ska 13y agore your edit: yes it was the latter interpretation that I had a problem with. The "group of sweets" == "crowd" doesn't really work for me, but I'll have to think about why.
- mrow84 13y agoYeah I only realised that you were interpreting it differently when I saw your reply to jimhefferon. If you do figure out what bothers you about the sweets==crowd interpretation then do please reply - statistics is so slippery that it takes a lot for me to feel fully confident with any particular idea.
- ska 13y agoOk, I think the simple version of this is: You are basically constructing a situation where the dominant bias is the same for all of the measurements (e.g. pairs of students). So you have a problem with your method of sampling being biased in a predictable way. So far, so good and this is an interesting lesson. But they are all using the same estimation method from this biased information, and they are all (by construction) creating the same bias, and that is very much not crowd-like behavior. Applying this as an example of the problem with "wisdom of crowds" seems to really miss the point to me, because you created a situation where they didn't behave like a crowd.
- wpietri 13y agoSeems fair to me. Note that he published all the estimates and then had people make predictions as a group, which is pretty much the way to extract wisdom-of-crowds effects. I agree the estimator is biased, but in good part because of how people pick 5 from 100. We are all biased in some ways. If a group shares the same biases, then a wisdom-of-crowds approach will yield a wrong answer in which everybody is confident. I think it's a fine lesson for people trying to work in this fashion.
- sp332 13y agoHe didn't have them estimate anything. They weighed things with a scale, then multiplied by 20.
- ska 13y agoBut they shouldn't all be doing the "same" thing!
- wpietri 13y agoFrom the instructions: "Explain that their goal is to estimate the total weight of all the candies in the bag. They can choose their 5 candies using any method–systematic sampling, random sampling, whatever. Whichever pair guesses closest to the true weight. they get the whole bag!" They knew their goal was to estimate the bag, and they were allowed to pick any five candies they wanted. If somebody picked in proportion to frequency, they'd be fine.
- baddox 13y ago
- GenerocUsername 13y agoMy chemistry teacher pulled a much better example of "Wisdom of crowds". We were told to measure exactly 25 ml of both water and alcohol, mix them, then report the volume. It was a competition to see which team got the closest to 50 ml... So we all reported our values on the board at the front for comparison. At the end, the team closest to 50 was declared "A Liar" because alcohol and water do not mix to the same volume as their constituent parts, thus the final volume should have been closer to 46ml and the team closest to 46 was awarded as the winner of the competition. Blew my 15 year old mind and I never biased my answers towards my bias again.