9 ms·
In addition to human graders, and purely automated graders, there's also the option of hybrid human-machine systems. Some aspects of the grading might be offlo
by mncharity 6y ago
In addition to human graders, and purely automated graders, there's also the option of hybrid human-machine systems.
Some aspects of the grading might be offloaded from humans, as the range of tasks and cases is often diverse. Or multiple less skilled humans substituted for more expensive ones.
In math for instance, there's no need for a human to see a blank answer, or a clearly correct one.
A system can assess graders as well as students. Might suggest to a professor which subset of gradings by graduate students could use a quality check. Using non-AI-complete statistics like "when this person grades late at night, and takes a long time on an answer, but doesn't write much, and the student is someone flagged as being at risk, then the odds increase that the professor will want to have seen it". Or perhaps particular cases are best handled by a particular grader. Or should always get an independent cross check of two graders, to reduce grading variance and bias.
That a hybrid system might hypothetically support good grading, does not imply that that's the direction which often-unfortunate incentives will drive the tech. Years ago I saw a paper, about the post-WWII defense industry investment in CNC tech. One argument was there were other options, like recording and playing back a skilled machinist, which might have been widely deployed decades earlier. But getting rid of the dependency on skilled workmen was a management objective. Creating cheap automated grading may be far more incentivized that attempting harder and less cheap good grading.
- throwawaygh 6y agoThis industrial model of human capital production consistently fails. It’s literally the root of everything that is broken about the us k12 education system. “Taylorism for education but with ‘The AI’” will end the same as the factory model for k12 that gave rise to the pathologies that critics like pg are so quick to point out. ‘The AI’ is just the 2020 version of the control theory metaphors that resulted in disasters in the mid 20th century. Ironic.