4 ms·

In CS courses, I always appreciated when we were given access to the grading scripts/unit tests used by course staff. It made that feedback loop immediate, and

In CS courses, I always appreciated when we were given access to the grading scripts/unit tests used by course staff. It made that feedback loop immediate, and unless you were intentionally doing something weird you usually knew exactly what grade you were getting for your submission. As a TA, it was funny to see the ways a few students would overfit those tests. In one extreme case I literally saw a student replace a complicated function definition with a if-else chain that just determined which of the 4 test cases it was being run on…

bsder 5y ago
As an CS instructor, my solution for that was to have a skeleton level of unit tests but to switch in a full suite once the deadline passed. Your grade was based on passing the full tests. Practically everyone who "played fair" got the same grade for both. Occasionally, I would add a test that tripped everybody up and I'd have to go see what happened. Anybody who overfitted, however, got crushed. My favorite assignment was always the next to last project (before end of semester deadlines start getting crushing). I created only a single unit test to verify the test suite runner was functional, and the students had to submit the rest of the pre-deadline unit tests. And I would switch in my full suite at the deadline. I told people that anybody who passed that suite would get bonus points that everybody was always bugging me for. It was always absolute chaos. I only ever had one student pass the switched in suite (he got some nice bonus points out of that--didn't need them, of course). I would then reopen the project for a week (I planned for it) to let everybody else clean up their work and resubmit. You could see and feel the difference in the students after that assignment. The fact that testing was an integral part of programming and that testing was, in and of itself, a difficult problem was a revelation--and not a particularly welcome one.
bee_rider 5y ago
I'm currently dealing with a kind of similar situation, and honestly I find it a little odd -- I get questions from students which I'm pretty sure are honest and asked in good faith, but they seem to think they are very close to having solved the problem, despite having just over-fitted most of the tests. It is possible that the problem is poorly written (I'm just a TA, I didn't write it, and it looks pretty clear to me), or it is possible that I'm just really gullible and they aren't actually asking in good faith (I'm a trusting person, but I've been doing this kind of work for a while and so I've seen most of the dishonest questions, this doesn't look like one to me). I dunno, I think I'll just chalk it up to the long tail of weirdness that can occur when dealing with a bunch of students.
- gonehome 5y ago
  My guess is CS1 is often the first time a student has been given a problem they have to actually reason about to solve (rather than follow a script). This is hard to understand and adapt to and stresses out students that have been trained (for years) to learn the expected script. You eventually get to this level in math, but only way later in post graduate work (unless you’re exceptional). I think there’s likely a benefit in explicitly tackling this directly for young/new students, it might help them see the bigger picture. At least the earnest ones struggling to do well anyway.

bsder 5y ago

As an CS instructor, my solution for that was to have a skeleton level of unit tests but to switch in a full suite once the deadline passed. Your grade was based on passing the full tests. Practically everyone who "played fair" got the same grade for both. Occasionally, I would add a test that tripped everybody up and I'd have to go see what happened. Anybody who overfitted, however, got crushed. My favorite assignment was always the next to last project (before end of semester deadlines start getting crushing). I created only a single unit test to verify the test suite runner was functional, and the students had to submit the rest of the pre-deadline unit tests. And I would switch in my full suite at the deadline. I told people that anybody who passed that suite would get bonus points that everybody was always bugging me for. It was always absolute chaos. I only ever had one student pass the switched in suite (he got some nice bonus points out of that--didn't need them, of course). I would then reopen the project for a week (I planned for it) to let everybody else clean up their work and resubmit. You could see and feel the difference in the students after that assignment. The fact that testing was an integral part of programming and that testing was, in and of itself, a difficult problem was a revelation--and not a particularly welcome one.

bee_rider 5y ago

I'm currently dealing with a kind of similar situation, and honestly I find it a little odd -- I get questions from students which I'm pretty sure are honest and asked in good faith, but they seem to think they are very close to having solved the problem, despite having just over-fitted most of the tests. It is possible that the problem is poorly written (I'm just a TA, I didn't write it, and it looks pretty clear to me), or it is possible that I'm just really gullible and they aren't actually asking in good faith (I'm a trusting person, but I've been doing this kind of work for a while and so I've seen most of the dishonest questions, this doesn't look like one to me). I dunno, I think I'll just chalk it up to the long tail of weirdness that can occur when dealing with a bunch of students.

gonehome 5y ago
My guess is CS1 is often the first time a student has been given a problem they have to actually reason about to solve (rather than follow a script). This is hard to understand and adapt to and stresses out students that have been trained (for years) to learn the expected script. You eventually get to this level in math, but only way later in post graduate work (unless you’re exceptional). I think there’s likely a benefit in explicitly tackling this directly for young/new students, it might help them see the bigger picture. At least the earnest ones struggling to do well anyway.