3 ms·

It isn't. "Most of these answers were not just wrong, but obviously wrong", according to the teacher. The chances of picking the worst possible answer is lower

by tgv 3y ago

It isn't. "Most of these answers were not just wrong, but obviously wrong", according to the teacher. The chances of picking the worst possible answer is lower than 0.2. If the .01 threshold is fair? It isn't if you assume that someone tried to pass the test by filling out the whole thing at random. If the students tried to pass by choosing the correct answer, >= 19 obviously wrong answers out of 45 seems fair enough.

SilverBirch 3y ago
I'm not saying the conclusion they're cheating is wrong - in fact the threshold is way to generous, but fundamentally the statistical model he's picked just isn't a good fit.
- tgv 3y ago
  What model would you propose? You don't know the likelihood in advance, do you? The only thing that could have been added (that I can think of!) is that the likelihood of cheating is not memoryless, as the binomial process assumes, but rather that a cheater would not copy a few answers, but very many. So the chance that a cheater picks the bad answer is > 0.2, the chance that someone who doesn't know the answer = 0.2, and the chance that someone who does < 0.2. But given the fact that we don't know the correct probabilities, nor the scale of cheating, and not even how often people take a random guess, makes the model a decent starting point, IMO, for getting a first estimate. After all, the teacher is not researching stochastic models of cheating, but just grading papers.
  - SilverBirch 3y ago
    I'm not sure there's a completely thorough way of proving cheating on a single measure of a relatively short test. If I wanted something more thorough I'd probably compare the mean score on the non-cheat questions vs the score on the cheat questions though. Let's say you provide the correct answers on the "leaked" test. If the score on the leaked questions is significantly higher than the score on the unseen questions then you can confidently say there was some cheating. At that point you'd probably interview the candidates on their tests rather than throw accusations.

by tgv 3y ago

SilverBirch 3y ago

I'm not saying the conclusion they're cheating is wrong - in fact the threshold is way to generous, but fundamentally the statistical model he's picked just isn't a good fit.

tgv 3y ago
What model would you propose? You don't know the likelihood in advance, do you? The only thing that could have been added (that I can think of!) is that the likelihood of cheating is not memoryless, as the binomial process assumes, but rather that a cheater would not copy a few answers, but very many. So the chance that a cheater picks the bad answer is > 0.2, the chance that someone who doesn't know the answer = 0.2, and the chance that someone who does < 0.2. But given the fact that we don't know the correct probabilities, nor the scale of cheating, and not even how often people take a random guess, makes the model a decent starting point, IMO, for getting a first estimate. After all, the teacher is not researching stochastic models of cheating, but just grading papers.
- SilverBirch 3y ago
  I'm not sure there's a completely thorough way of proving cheating on a single measure of a relatively short test. If I wanted something more thorough I'd probably compare the mean score on the non-cheat questions vs the score on the cheat questions though. Let's say you provide the correct answers on the "leaked" test. If the score on the leaked questions is significantly higher than the score on the unseen questions then you can confidently say there was some cheating. At that point you'd probably interview the candidates on their tests rather than throw accusations.