3 ms·

0.1 standard deviation isn't significant. So for the most part it doesn't work. Personally I think it is fairly obvious why. No one does a bad job on purpose

0.1 standard deviation isn't significant. So for the most part it doesn't work. Personally I think it is fairly obvious why. No one does a bad job on purpose so if you aren't doing well then you are just not capable. This can be caused by being incompetent or you don't have the power/tools/money necessary to get the job done. Offering a small bonus doesn't really change either one. Sure there may be some motivational issues, but they would have to be systemic for this sort of ploy to work.

cwan 17y ago
You may have misread the quote. It's more than a 0.1 standard deviation: "At the end of two years of the program, students in incentive schools performed significantly better than those in comparison schools by 0.28 and 0.16 standard deviations (SD) in math and language tests respectively...." That's 0.28 for math and 0.16 for language. I'd say that's pretty significant - especially for a large sample size. In this case presumably because of the sample size it's more than being about "power/tools/money".
- stonemetal 17y ago
  From wikipedia: In science, researchers commonly report the standard deviation of experimental data, and only effects that fall far outside the range of standard deviation are considered statistically significant—normal random error or variation in the measurements is in this way distinguished from causal variation. Thus the results are not statistically significant aka with in normal random error etc. So when did 0.3 of not significant become significant.
  - yummyfajitas 17y ago
    You are confusing the standard deviation of the sample mean with the standard deviation of the sample. http://en.wikipedia.org/wiki/Central_limit_theorem http://en.wikipedia.org/wiki/Central_limit_theorem
  - cwan 17y ago
    My skills in stats aren't quite current but this was from a large sample size - covering 500 schools and more than 55,000 students. From the wikipedia article on statistical significance (http://en.wikipedia.org/wiki/Statistical_significance http://en.wikipedia.org/wiki/Statistical_significance): "Given a sufficiently large sample, extremely small and non-notable differences can be found to be statistically significant, and statistical significance says nothing about the practical significance of a difference." This isn't even "extremely small". From the report: "The mean treatment effect of 0.22 SD is equal to 9 percentile points at the median of a normal distribution. We find a minimum average treatment effect of 0.1 SD at every percentile of baseline test scores, suggesting broad-based gains in test scores as a result of the incentive program."

cwan 17y ago

You may have misread the quote. It's more than a 0.1 standard deviation: "At the end of two years of the program, students in incentive schools performed significantly better than those in comparison schools by 0.28 and 0.16 standard deviations (SD) in math and language tests respectively...." That's 0.28 for math and 0.16 for language. I'd say that's pretty significant - especially for a large sample size. In this case presumably because of the sample size it's more than being about "power/tools/money".

stonemetal 17y ago
From wikipedia: In science, researchers commonly report the standard deviation of experimental data, and only effects that fall far outside the range of standard deviation are considered statistically significant—normal random error or variation in the measurements is in this way distinguished from causal variation. Thus the results are not statistically significant aka with in normal random error etc. So when did 0.3 of not significant become significant.
- yummyfajitas 17y ago
  You are confusing the standard deviation of the sample mean with the standard deviation of the sample. http://en.wikipedia.org/wiki/Central_limit_theorem http://en.wikipedia.org/wiki/Central_limit_theorem
- cwan 17y ago
  My skills in stats aren't quite current but this was from a large sample size - covering 500 schools and more than 55,000 students. From the wikipedia article on statistical significance (http://en.wikipedia.org/wiki/Statistical_significance http://en.wikipedia.org/wiki/Statistical_significance): "Given a sufficiently large sample, extremely small and non-notable differences can be found to be statistically significant, and statistical significance says nothing about the practical significance of a difference." This isn't even "extremely small". From the report: "The mean treatment effect of 0.22 SD is equal to 9 percentile points at the median of a normal distribution. We find a minimum average treatment effect of 0.1 SD at every percentile of baseline test scores, suggesting broad-based gains in test scores as a result of the incentive program."