3 ms·
The judgement criteria itself looks step-like for many of these things. Word unscrambling, for instance: does a 15 percentage point increase in accuracy sugges
by majormajor 4y ago
The judgement criteria itself looks step-like for many of these things.
Word unscrambling, for instance: does a 15 percentage point increase in accuracy suggest that you're 15x better at it (from close to 0 to 15%ish)? Or could you do that by being modestly better, but that's just enough to cross the threshold to solve the simplest 15% of words in the task challenge you're being graded on?
We're also using a log scale on the X axis there but a linear one for the Y axis which amplifies the "steppiness"
- bob1029 4y agoThis is a fair point. Things like arithmetic either work or they don't. Is partial credit issued as a linear interpolation between the predicted value and expected value? If not, then you necessarily will have step-like behavior in the data.