5 ms·
> We routinely rely on automated assessment to evaluate our students’ work on programming assignments. Wait, why would you do that? Algorithms are going to be
by gregmac 8y ago
> We routinely rely on automated assessment to evaluate our students’ work on programming assignments.
Wait, why would you do that? Algorithms are going to be terrible at judging the hardest and most important part of writing code: how maintainable it is.
Garbage code, through brute force, can eventually pass all test cases -- I hope that is not what schools are teaching.
The idea of using test cases isn't a bad way to form part of an evaluation, given the code getting worked on has a defined interface spec. It can also be done the other way around: run code that produces various bad results to exercise the student's test cases.
However, more often than not in the real world specs don't exist at this level, and by using them, they are going to lead people to certain implementations (rather than showing whether the student can come up with something appropriate on their own). From my experience in school, they often also force a compromise design because the imposed interface has shortcomings (eg: functions that use several output parameters rather than returning an object, or force a certain functional composition that is entirely different than what I would do and how I think).
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- skrishnamurthi 8y agoCo-author here. 1. It's PART OF the assessment. There's also a ton of human review. But humans aren't tops at checking for correctness. Using automation helps us run against a large set of tests, both normal and corner cases, and helps us figure out when things have gone wrong. Can even help the human find things they may have missed and need to re-review. 2. Also, some systems, like many MOOCs, use largely or only automated methods for "scalability" reasons. I don't defend this practice, but it's the reality. Our view is that those who are doing that need to be even more careful about making sure their practices make sense. 3. > It can also be done the other way around: run code that produces various bad results to exercise the student's test cases. Which is something the article is actually about (eye roll). Sometimes helps to read beyond the first sentence.
- mattnewton 8y agoAs a former TA when I was in undergrad, having an initial filter of “does this compile and produce output” probably regularly cut 25% of the submissions out. If that isn’t working, it wasn’t worth my time to grade. I think that’s a real world enough stance.
- userbinator 8y agoWith a similar experience as you, I agree. An absolute minimum bar is to have it compile, and tests will help in ascertaining correctness beyond that. Those which pass all tests only need a quick glance at the code to determine if it is reasonable.
- gowld 8y agoThat's an awful attitude for a teacher. Why are you even there if you only want to spend time on students who don't need your services?
- Retra 8y agoYou're reading that statement far too uncharitably. They said nothing at all about teaching, and only mentioned how students interact with one small system. Obviously that's not the whole job?
- icebraining 8y agoIn my uni, the test suite was public; you were supposed to run it and talk to the TA about the tests you can't pass before the deadline. If you didn't do that, and just submitted broken code, then you'd fail the test. Seems fair to me.
- dang 8y ago"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- skrishnamurthi 8y agoWell, it depends on how the TA's job is defined. If it's to also help students, then code that doesn't pass is very much worth grading. Indeed, the failures may be useful in determining where the student needs help.
- kevinpet 8y agoMaintainability is not the most important part of writing code. It becomes the most important thing to try to improve once you get to the point that your code is correct, well tested, and has acceptable performance.