4 ms·

> Low scores are very susceptible to noise (the delta between 50% and 60% is greater than between 85% and 95%) This seems untrue from information theory. The

> Low scores are very susceptible to noise (the delta between 50% and 60% is greater than between 85% and 95%) This seems untrue from information theory. The entropy of a question is maximized when its probability of being answered correctly is exactly 50% [1]. If your only goal is to have the least amount of measurement noise given a fixed number of questions, then you'll want each question to be hard enough to filter 50% of the class out, and to minimize the correlation between questions. For example, 10 independent 50% questions reveals as much information as 16 independent 85% questions. > Imagine I gave this as an exam to a math class of 20 students. One person scores a 1. The rest score 0. Is it meaningful to curve this? You're looking at only 1 tail but ignoring the other. By symmetry, a question that only 1 person gets correct tells you exactly as much as a question that 19 get correct. An exam with a 90% pass rate (before curving) is no better than a 10% exam. [1] https://en.wikipedia.org/wiki/Binary_entropy_function https://en.wikipedia.org/wiki/Binary_entropy_function

mcguire 10y ago
Thanks. I'm now having flashbacks to the summer of 1990, when I worked on a Education professor's research project, creating a math test in a Hypercard stack that presented the question that provided the most information about the testee's abilities, given their previous answers.
- smallnamespace 10y ago
  The computerized GMAT and LSAT attempt to do this - the difficulty level of questions dynamically adjust to your current performance level (higher means harder questions).
BeetleB 10y ago
First, while you may have a point, you misunderstood my percentages. I did not mean them to be the pass rate, but rather the score. > An exam with a 90% pass rate (before curving) is no better than a 10% exam. I do not encourage a 90% pass rate. I do not endorse exams where most of the students score 90%. I'm saying an exam which allows for a large variation in scores (anywhere from 10 to 100) yields more information, whereas an exam where a really brilliant student will get 50, with the next highest score being a 30 by someone who is only very smart, tends to be less likely to yield useful information about most of the students who will get 20 or less. A fairly bright student and a fairly average student may both score a 20 on such a test - yet the test failed to distinguish between them. (BTW, I had an instructor whose exams were like this - I think I once had the highest score at around 25-30 out of 100). A less demanding, but not trivial test, will separate out the average from the brighter. Regardless, why the need for a curve? Your grading system should not depend on which students are present. It can lead to poor students getting a good grade and smarter students getting a poorer grade - in different semesters, for the very same tests.
- smallnamespace 10y ago
  I was unclear before -- an exam with an average score of 90 is equally bad as one with an average score of 10. The variation in score is maximized when the exam has an average score of 50. Since an uncurved score of 50 is a failing grade, you'd need curving to make that system work.
  - BeetleB 10y ago
    >I was unclear before -- an exam with an average score of 90 is equally bad as one with an average score of 10. I agree but it's not what I'm advocating for. >Since an uncurved score of 50 is a failing grade, you'd need curving to make that system work. A curve is inherently about grading relative to one's peers, and I see no reason why it is required. If your test will have an average score of 50, make that a B (or whatever), and perhaps 75 and A. Just do it and fix it to those grades. Do not keep changing the threshold based on your current batch of students. I think the fundamental differences between the two camps boils down to this question: Should grades be an absolute measure or a relative measure? I'm strongly in the absolute camp.
    - smallnamespace 10y ago
      > If your test will have an average score of 50, make that a B (or whatever), and perhaps 75 and A. Just do it and _fix_ it to those grades. That's still a form of curving, since you can't know ahead of time on a new test how students will do. E.g. if you arbitrarily fixed it at 50 / 75 and your students do a lot worse than you expected, do you just fail the lot of them? If not, and you move the score thresholds lower, then that's just what curving is. In fact, I had a couple of professors who told us ahead of time that curving could only be used to adjust our scores upward, but never downward, e.g. 70% was passing, but so was 60%, if enough of the class did poorly. Made the students feel much better, since they had a fixed bar to reach. Seems like you'd be satisfied though if the curve used at least ~100 students to calculate so that it didn't change much due to random variation (e.g. if your class is small, aggregate multiple years of data)? Still a form of curving (since you are, at the end of the day, being compared to other students), just with different methodology.

mcguire 10y ago

Thanks. I'm now having flashbacks to the summer of 1990, when I worked on a Education professor's research project, creating a math test in a Hypercard stack that presented the question that provided the most information about the testee's abilities, given their previous answers.

smallnamespace 10y ago
The computerized GMAT and LSAT attempt to do this - the difficulty level of questions dynamically adjust to your current performance level (higher means harder questions).

BeetleB 10y ago

First, while you may have a point, you misunderstood my percentages. I did not mean them to be the pass rate, but rather the score. > An exam with a 90% pass rate (before curving) is no better than a 10% exam. I do not encourage a 90% pass rate. I do not endorse exams where most of the students score 90%. I'm saying an exam which allows for a large variation in scores (anywhere from 10 to 100) yields more information, whereas an exam where a really brilliant student will get 50, with the next highest score being a 30 by someone who is only very smart, tends to be less likely to yield useful information about most of the students who will get 20 or less. A fairly bright student and a fairly average student may both score a 20 on such a test - yet the test failed to distinguish between them. (BTW, I had an instructor whose exams were like this - I think I once had the highest score at around 25-30 out of 100). A less demanding, but not trivial test, will separate out the average from the brighter. Regardless, why the need for a curve? Your grading system should not depend on which students are present. It can lead to poor students getting a good grade and smarter students getting a poorer grade - in different semesters, for the very same tests.

smallnamespace 10y ago
I was unclear before -- an exam with an average score of 90 is equally bad as one with an average score of 10. The variation in score is maximized when the exam has an average score of 50. Since an uncurved score of 50 is a failing grade, you'd need curving to make that system work.
- BeetleB 10y ago
  >I was unclear before -- an exam with an average score of 90 is equally bad as one with an average score of 10. I agree but it's not what I'm advocating for. >Since an uncurved score of 50 is a failing grade, you'd need curving to make that system work. A curve is inherently about grading relative to one's peers, and I see no reason why it is required. If your test will have an average score of 50, make that a B (or whatever), and perhaps 75 and A. Just do it and fix it to those grades. Do not keep changing the threshold based on your current batch of students. I think the fundamental differences between the two camps boils down to this question: Should grades be an absolute measure or a relative measure? I'm strongly in the absolute camp.
  - smallnamespace 10y ago
    > If your test will have an average score of 50, make that a B (or whatever), and perhaps 75 and A. Just do it and _fix_ it to those grades. That's still a form of curving, since you can't know ahead of time on a new test how students will do. E.g. if you arbitrarily fixed it at 50 / 75 and your students do a lot worse than you expected, do you just fail the lot of them? If not, and you move the score thresholds lower, then that's just what curving is. In fact, I had a couple of professors who told us ahead of time that curving could only be used to adjust our scores upward, but never downward, e.g. 70% was passing, but so was 60%, if enough of the class did poorly. Made the students feel much better, since they had a fixed bar to reach. Seems like you'd be satisfied though if the curve used at least ~100 students to calculate so that it didn't change much due to random variation (e.g. if your class is small, aggregate multiple years of data)? Still a form of curving (since you are, at the end of the day, being compared to other students), just with different methodology.