4 ms·

> you have to shuffle the order in which you grade these submissions each week, for fairness I don't think this is fair. It's just a more randomly distributed

> you have to shuffle the order in which you grade these submissions each week, for fairness I don't think this is fair. It's just a more randomly distributed unfairness, rather than by a deterministic factor (like the student's name) 'Fair' would be each student is assessed independently for the work they did, rather than their mark being impacted by how early or late they were marked.

luplex 3y ago
There are many notions of "fairness", many of which are logically incompatible with each other. In this example, I think it's kind of fair to give everyone an equal chance of being advantaged. You're not hurting anyone specifically.
shepherdjerred 3y ago
It would be essentially impossible to have something "truly" fair for open-ended questions since humans are stateful. Maybe this is a case that AI could actually do quite well. Manually grade the answers and identify the classes of mistakes. Then hand the classes of mistakes to the AI and ask for it to determine which answers have which types of mistakes. Once you've done that, you just need to associate a deduction for each type of mistake and do some simple math.
- vagrantJin 3y ago
  what do you mean AI? you must be joking.
  - shepherdjerred 3y ago
    Imagine a question: compare bubble sort and quick sort algorithm. Some students might mix up the algorithms, some might give the incorrect computation complexity, some might describe them incorrectly in some way. Manually grade some (or all of) the answers by noting the kinds of things students got wrong (e.g. the above criteria). Then, feed in to ChatGPT (or your favorite alternative) the answer + the categories of mistakes to expect. Here's a simplified example: https://chat.openai.com/share/bf801e12-51d5-4255-9968-bbf91bb29196 https://chat.openai.com/share/bf801e12-51d5-4255-9968-bbf91b...
gqcwwjtg 3y ago
Is that distinction worth making here? There’s no way to “assess independently” the work of each student without some amount of randomness. But I think that’s okay, because isn’t randomly distributed unfairness just… fairness?
jcparkyn 3y ago
I think an important difference is that when you shuffle them, the unfairness stops being correlated across multiple assignments, so the "aggregate" unfairness over the course of the semester is much lower.

luplex 3y ago

There are many notions of "fairness", many of which are logically incompatible with each other. In this example, I think it's kind of fair to give everyone an equal chance of being advantaged. You're not hurting anyone specifically.

shepherdjerred 3y ago

It would be essentially impossible to have something "truly" fair for open-ended questions since humans are stateful. Maybe this is a case that AI could actually do quite well. Manually grade the answers and identify the classes of mistakes. Then hand the classes of mistakes to the AI and ask for it to determine which answers have which types of mistakes. Once you've done that, you just need to associate a deduction for each type of mistake and do some simple math.

vagrantJin 3y ago
what do you mean AI? you must be joking.
- shepherdjerred 3y ago
  Imagine a question: compare bubble sort and quick sort algorithm. Some students might mix up the algorithms, some might give the incorrect computation complexity, some might describe them incorrectly in some way. Manually grade some (or all of) the answers by noting the kinds of things students got wrong (e.g. the above criteria). Then, feed in to ChatGPT (or your favorite alternative) the answer + the categories of mistakes to expect. Here's a simplified example: https://chat.openai.com/share/bf801e12-51d5-4255-9968-bbf91bb29196 https://chat.openai.com/share/bf801e12-51d5-4255-9968-bbf91b...

gqcwwjtg 3y ago

Is that distinction worth making here? There’s no way to “assess independently” the work of each student without some amount of randomness. But I think that’s okay, because isn’t randomly distributed unfairness just… fairness?

jcparkyn 3y ago

I think an important difference is that when you shuffle them, the unfairness stops being correlated across multiple assignments, so the "aggregate" unfairness over the course of the semester is much lower.