3 ms·

Are Oklahoma Students Really This Dumb? Or Is Strategic Vision ...

34 points by riffer 17y ago

CrazedGeek 17y ago
An analysis of the post: http://blog.smellthedata.com/2009/09/analysis-of-pollster-fraud-and-oklahoma.html http://blog.smellthedata.com/2009/09/analysis-of-pollster-fr...
- yannis 17y ago
  Both posts are very well thought out. I always mistrust polls and any statistics that don't publish details of their methodology and raw data. Sometime back I spent some involuntary time in the army. We had to take a number of tests. I answered most of them randomly. On the IQ test, I really tried h a r d to get all the questions wrong, in the vain that they will think I was an idiot and they let me go (they never did and no-one ever gave me a strange look). I sympathize with those Oklahoma kids and understand that there many ways to tell a grown-up a 'fuck you' ;)
  - CrazedGeek 17y ago
    As an Oklahoman high-schooler, I think the "they made the whole thing up" hypothesis is way more likely. Either that or they got a really unrepresentative sample. I know that at least half of my 2000 student school could have made at least a 6 on that quiz. Not too many people here actively try to buck the system, sadly.
    - TetOn 17y ago
      I tend to agree, though my feelings fall more on getting student buy-in. Without some real outcome that matters (a grade, or something similar) what's to insure a reliable sample. Assuming things haven't changed since I was in high school, I'm pretty sure we'd have gone out of our way to skew the test purposely with outrageously wrong answers...
chrischen 17y ago
"communist and republican" ... lol those kids got brainwashed!
- jrockway 17y ago
  The average American high-school student is dumb, but not that dumb.
- andreyf 17y ago
  The question is a bit of an affront to one's intelligence, so it makes sense to pick the silly answer ;)
  - dagw 17y ago
    I remember when I was in high school and they where doing some sort of poll about the upcoming nation election at several school. Somehow word started to spread around that we should all 'vote' for the Communists, so we did. A few days later there was an article in the local newspaper about a disturbing trend showing the over 20% of 16-18 year olds would have voted communist and what this might mean for the next election when all these kids have the right to vote. We found the whole thing highly amusing.
btilly 17y ago
From the answers to the questions I am inclined to believe in the "made it all up" theory. But at a 99% confidence level I can reject the possibility that it was made up with a simple simulation of the type described. How am I so sure? Well we're given all of the probabilities for all of the answers. We can therefore calculate what (assuming independence) the likely odds are of different numbers of correct answers. We're also given the number of students who got each number correct. Here is the result (I'll give the program that I used at the end): 0 right: 46 vs 32.56 1 right: 158 vs 147.61 2 right: 246 vs 273.73 3 right: 265 vs 278.71 4 right: 177 vs 174.46 5 right: 80 vs 70.58 6 right: 22 vs 18.74 7 right: 6 vs 3.23 8 right: 0 vs 0.35 9 right: 0 vs 0.02 10 right: 0 vs 0.00 The argument is that we should be suspicious because those two sets of numbers are too similar. The tails are not heavy enough. Now I admit that graphically it does look close. But if you look, the tails _are_ heavy. Is that difference significant? We can use a confidence testing to answer that. The null hypothesis is that the distribution comes from a simple simulation and should be close to the theoretical numbers that I just gave. The alternate hypothesis is that the tails should be heavier and the middle lower. Let's look at the middle because it is easier to calculate. Theoretically 0.55244 of the test population should get one of those two numbers correct. It is claimed that 511/1000 did. How likely is that? Well in the null hypothesis each student is an independent observation. Each observation is a 1 or 0 with average 0.55244. The variance of an observation is readily calculated and turns out to be 0.2472500464. The number of people getting one of those two answers after 1000 observations is therefore approximately normal with average 552.44 and variance 1000 * 0.2472500464 = 247.2500464. The standard deviation is the square root of that, which is 15.724. That puts the observed value more than 2.63 standard deviations out. at a 99% confidence we can conclude that the numbers were NOT generated by a simple simulation of the type described. For the curious, here is how I calculated the theoretical distribution of answers: #! /usr/bin/perl -w use strict; use Math::Polynomial; my $ans = 1; # Number from the % of students who got each question right. for my $x (qw(.28 .26 .27 .1 .14 .61 .43 .11 .23 .29)) { $ans *= Math::Polynomial->new(1-$x, $x); } print "$ans\n"; __END__ (5.295902595984e-07 x^10 + 2.07154949414e-05 x^9 + 0.000345419801607472 x^8 + 0.00322944050567802 x^7 + 0.0187445906666514 x^6 + 0.0705826597827996 x^5 + 0.174464060957845 x^4 + 0.278709766012616 x^3 + 0.273731283943492 x^2 + 0.147609870020204 x + 0.0325616632239042)

34 points by riffer 17y ago

An analysis of the post: http://blog.smellthedata.com/2009/09/analysis-of-pollster-fraud-and-oklahoma.html http://blog.smellthedata.com/2009/09/analysis-of-pollster-fr...

yannis 17y ago
Both posts are very well thought out. I always mistrust polls and any statistics that don't publish details of their methodology and raw data. Sometime back I spent some involuntary time in the army. We had to take a number of tests. I answered most of them randomly. On the IQ test, I really tried h a r d to get all the questions wrong, in the vain that they will think I was an idiot and they let me go (they never did and no-one ever gave me a strange look). I sympathize with those Oklahoma kids and understand that there many ways to tell a grown-up a 'fuck you' ;)
- CrazedGeek 17y ago
  As an Oklahoman high-schooler, I think the "they made the whole thing up" hypothesis is way more likely. Either that or they got a really unrepresentative sample. I know that at least half of my 2000 student school could have made at least a 6 on that quiz. Not too many people here actively try to buck the system, sadly.
  - TetOn 17y ago
    I tend to agree, though my feelings fall more on getting student buy-in. Without some real outcome that matters (a grade, or something similar) what's to insure a reliable sample. Assuming things haven't changed since I was in high school, I'm pretty sure we'd have gone out of our way to skew the test purposely with outrageously wrong answers...

chrischen 17y ago

"communist and republican" ... lol those kids got brainwashed!

jrockway 17y ago
The average American high-school student is dumb, but not that dumb.
andreyf 17y ago
The question is a bit of an affront to one's intelligence, so it makes sense to pick the silly answer ;)
- dagw 17y ago
  I remember when I was in high school and they where doing some sort of poll about the upcoming nation election at several school. Somehow word started to spread around that we should all 'vote' for the Communists, so we did. A few days later there was an article in the local newspaper about a disturbing trend showing the over 20% of 16-18 year olds would have voted communist and what this might mean for the next election when all these kids have the right to vote. We found the whole thing highly amusing.

btilly 17y ago

From the answers to the questions I am inclined to believe in the "made it all up" theory. But at a 99% confidence level I can reject the possibility that it was made up with a simple simulation of the type described. How am I so sure? Well we're given all of the probabilities for all of the answers. We can therefore calculate what (assuming independence) the likely odds are of different numbers of correct answers. We're also given the number of students who got each number correct. Here is the result (I'll give the program that I used at the end): 0 right: 46 vs 32.56 1 right: 158 vs 147.61 2 right: 246 vs 273.73 3 right: 265 vs 278.71 4 right: 177 vs 174.46 5 right: 80 vs 70.58 6 right: 22 vs 18.74 7 right: 6 vs 3.23 8 right: 0 vs 0.35 9 right: 0 vs 0.02 10 right: 0 vs 0.00 The argument is that we should be suspicious because those two sets of numbers are too similar. The tails are not heavy enough. Now I admit that graphically it does look close. But if you look, the tails _are_ heavy. Is that difference significant? We can use a confidence testing to answer that. The null hypothesis is that the distribution comes from a simple simulation and should be close to the theoretical numbers that I just gave. The alternate hypothesis is that the tails should be heavier and the middle lower. Let's look at the middle because it is easier to calculate. Theoretically 0.55244 of the test population should get one of those two numbers correct. It is claimed that 511/1000 did. How likely is that? Well in the null hypothesis each student is an independent observation. Each observation is a 1 or 0 with average 0.55244. The variance of an observation is readily calculated and turns out to be 0.2472500464. The number of people getting one of those two answers after 1000 observations is therefore approximately normal with average 552.44 and variance 1000 * 0.2472500464 = 247.2500464. The standard deviation is the square root of that, which is 15.724. That puts the observed value more than 2.63 standard deviations out. at a 99% confidence we can conclude that the numbers were NOT generated by a simple simulation of the type described. For the curious, here is how I calculated the theoretical distribution of answers: #! /usr/bin/perl -w use strict; use Math::Polynomial; my $ans = 1; # Number from the % of students who got each question right. for my $x (qw(.28 .26 .27 .1 .14 .61 .43 .11 .23 .29)) { $ans *= Math::Polynomial->new(1-$x, $x); } print "$ans\n"; __END__ (5.295902595984e-07 x^10 + 2.07154949414e-05 x^9 + 0.000345419801607472 x^8 + 0.00322944050567802 x^7 + 0.0187445906666514 x^6 + 0.0705826597827996 x^5 + 0.174464060957845 x^4 + 0.278709766012616 x^3 + 0.273731283943492 x^2 + 0.147609870020204 x + 0.0325616632239042)