5 ms·
> I feel like you'd probably have to have an AGI to meaningfully evaluate an essay. So the reason this isn't the case, is because there are very simple metrics
by dcolkitt 7y ago
> I feel like you'd probably have to have an AGI to meaningfully evaluate an essay.
So the reason this isn't the case, is because there are very simple metrics that tend to highly correlate with essay quality. It doesn't mean the grading-bot is actually evaluating essay quality. It's just looking for properties that are statistically associated with good essays. Remember, at the end of the day as long as the bot's ranking is close enough to the human grader's ranking, nobody really cares about the internal logic.
A very straightforward example is spelling mistakes. People who make spelling mistakes aren't necessarily bad writers. And vice versa, there may be great speller who can't write for shit. But by and large the people who spell poorly also tend to write poorly. Easily detectable grammatical issues, like misplaced modifiers, subject verb disagreement, or inconsistent tense, are also correlated indicators.
A very simple metric is essay length. Especially if its a timed exam. Good writers tend to have verbal fluidity, with words easily flowing to paper. They don't struggle converting thoughts too sentences. So they tend to end up with the most words written down within a fixed time period. By and large the longer a timed essay is, the more likely that its actual quality is high.
Grading bots basically rely on these statistical relationships. They're not measuring anything intrinsic to good writing. But at the end of the day, their student rankings are usually pretty close to that of a typical human grader. In some cases the bot will have a closer ranking to a random human grader, than two random human graders will have to each other.
The biggest flaw here is Goodwin's law. When the test takers become aware of the kludges that the bots use, they can exploit it. For example just dump a bunch of verbal diarrhea with as many correctly spelled words as possible. But even then it doesn't really hurt the bot's ranking accuracy too much. Because the kids who do the most test-prep and learn all the tips and tricks, are usually high-achievers who do well on essays anyway.
- bo1024 7y agoStrongly (but respectfully) disagree with a lot of this! This is related to current fairness-in-AI discussions. In many cases the basic problem is ML systems leverage correlations for making causal decisions. Here, there is a huge ethical difference between scoring a person based on "is this a good essay" and "do the features of this essay correlate with features of good essays". Just like there is a huge fairness and discrimination difference between "is this person qualified for a loan" and "do the features of this person correlate with features of people who qualify for loans" (algorithmic redlining). Your last sentence has a big discrimination/fairness issue also, since you are testing even more for parental income and parental free time.
- nerdponx 7y agoWhich means that machine learning models need to not only be good predictors, but also be causal models to some extent.
- ptx 7y agoI think you meant Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."
- JoeAltmaier 7y agoIs that what it's called? Its referred in neural net training as overtraining.
- dcolkitt 7y agoHaha, whoops. That mistake kind of changes the meaning... Thanks for catching the mistake.
- mannykannot 7y agoYour last paragraph, and particularly the last sentence, epitomizes what is wrong with your whole thesis: the ultimate goal of the testing (and education itself, for that matter) is not to find people who can "do well on essays"; it is to develop analytical thinking.
- JoeAltmaier 7y agoClearly, that referred to 'doing well on essays' correlating to good analytical thinkers. Which is does.
- mannykannot 7y agoThat assumption lacks justification when the scoring does not actually measure analytical thinking. Any statistical evidence for it is suspect as a predictor of future outcomes when a high score can more easily be gamed than 'honestly' achieved.
- JoeAltmaier 7y agoScoring is not the point here; the analytical thinker is gaming the test to pump the score, thus proving they are an analytical thinker. Not a statistical argument; a suggestion that the screen works, when it is abused. Because it is abused. Anyway, yeah, not really a correlation.
- 6gvONxR4sf7o 7y agoI can't disagree strongly enough. >Remember, at the end of the day as long as the bot's ranking is close enough to the human grader's ranking, nobody really cares about the internal logic. This isn't true at all. Imagine you got a B or C on an essay that a human would have given an A to because you wrote it concisely and in plain language, or because you used language that's statistically correlated with being black. Does the fact that this is rare console you? "Sorry, but it's usually very close to the human grader's ranking." Close enough isn't good enough when you get the short end of the stick. "Sorry, you aren't going to get to go to the college you wanted because you use language statistically correlated with poor writing." Or just because you're different, so the statistical correlation doesn't apply to you, you filthy outlier. Just because it's a rare event doesn't make it okay. In adulthood, this is like hiring or firing for work statistically correlated with good work. Remember when amazon rolled out the resume scorer? [0] Sure it was biased towards women, but it was close enough to human scores, so who cares about the internal logic? >Grading bots basically rely on these statistical relationships. They're not measuring anything intrinsic to good writing. At the end of the day, our goal here is to measure good writing. If the bots aren't measuring anything intrinsic to good writing, we shouldn't use them. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G https://www.reuters.com/article/us-amazon-com-jobs-automatio...
- LorenPechtel 7y agoThe problem with the bots is that while they average agreement with the humans they can produce very different results. Fine if you're seeing how a school is doing, horrible if you're testing how a student is doing.
- davidhowlett 7y agoMinor correction; The automated resume reviewer was biased against women according to your reference.