4 ms·

I upvoted you for the interesting line of thought, although I disagree with the premise that acceptability of grading algorithmically implies acceptability of w

I upvoted you for the interesting line of thought, although I disagree with the premise that acceptability of grading algorithmically implies acceptability of writing algorithmically. In CS, we often grade assignments automatically, although we expect students to write the assignments by hand. Although there are many classes where this would be a bad idea, there are some where it works quite well - if the problems are strictly specified, grading an assignment with what are effectively unit tests is reasonable. Grading and doing an assignment are different classes of problems - roughly mapping to decision problems and search problems. Just because the grading is computationally tractable or acceptable does not necessarily imply that the act of writing the assignment is. I agree with your second paragraph by and large. There is a time and place for automatic grading, and most writing should not fall into that category.

_delirium 14y ago
One difference with CS algorithmic grading is that it's much more specific and causally connected to outcomes. It's not looking for statistical correlates of the input text and a notion of "good program", but is actually finding things like compile errors and failed testcases. And, crucially, it can also report those to the student: your program is wrong because when fed 5 it gives an output of 23 when the output should be 25. It's probably possible to grade student programs like this essay-grading system works: instead of feeding the programs to a compiler and running against test-cases, just turn the source code into a bag-of-words representation and feed it to a statistical classifier. There are almost certainly some statistical correlates between the source code of programs and whether they're good/bad submissions. But the results would be both less reliable and less informative as feedback.
- jacktoole1 14y ago
  Good point; I agree this makes code much more naturally machine graded than essays, and a statistical classifier for source code sounds disturbing (although it might do better on grading style than the usual autograding does).

_delirium 14y ago

One difference with CS algorithmic grading is that it's much more specific and causally connected to outcomes. It's not looking for statistical correlates of the input text and a notion of "good program", but is actually finding things like compile errors and failed testcases. And, crucially, it can also report those to the student: your program is wrong because when fed 5 it gives an output of 23 when the output should be 25. It's probably possible to grade student programs like this essay-grading system works: instead of feeding the programs to a compiler and running against test-cases, just turn the source code into a bag-of-words representation and feed it to a statistical classifier. There are almost certainly some statistical correlates between the source code of programs and whether they're good/bad submissions. But the results would be both less reliable and less informative as feedback.

jacktoole1 14y ago
Good point; I agree this makes code much more naturally machine graded than essays, and a statistical classifier for source code sounds disturbing (although it might do better on grading style than the usual autograding does).