4 ms·

The author completely misses here - probably due to limited exposure to real-life third-party code in real life production systems. Code auto-grading, at least

by trekkin 14y ago

The author completely misses here - probably due to limited exposure to real-life third-party code in real life production systems. Code auto-grading, at least at Coursera, is usually done by running comprehensive unit tests, which extensively test border cases as well. These test suites are often 5-10 times larger than the actual submitted code, and it is difficult to imagine anybody outside of this type of environment spending so much extra time designing (and testing!) test suites with 100% coverage. Moreover, code submissions have to comply with (or implement, in case of Java) predefined interfaces. And some courses (e.g. Scala) have style checker output taken into account (20% of grade is decided by the style checker in the Scala course). In summary, well-thought-out test suites and interface specifications demand well-designed code submissions; in real life, poor comments or sloppy expressions are a very minor nuisance compared to poorly designed interfaces and forgotten border cases.

gu 14y ago
I wasn't talking about insufficient test cases. Your remark about extensive unit tests is therefore quite irrelevant in this context as I don't question at all that the unit tests take border cases into account. I am mostly concerned with "soft" aspects. Just consider the case where a student has to define variables, but picks variable names in a language other than English, or where control flow in a submission is more convoluted than it would have to be. Those are the cases I discuss in the article. Moments ago, someone left a very fitting comment on my blog: "I am taking the edX CS169.1 course and I find that I will consistently have a "less than elegant" solution that the auto grader accepts but that I feel is sub-par. The irony is this class has a large BDD/TDD aspect and is teaching RED-GREEN-REFACTOR, but with an auto grader once its green there is little reason to go back and refactor."
- trekkin 14y ago
  Let me try to make my point by a "soft" analogy, in your terminology. If a fiction book has a great, gripping plot and interesting, relatable, wonderfully done characters, then weird spelling and heavy sentences are not a big deal and can be easily fixed by a competent editor. But nothing can save a very grammatical and clean-written text that is just flat, boring, or makes no sense at all. Ask any publisher - which kind of books they prefer? Similarly in software, getting the big picture right is much much more important than "elegance" in each individual line.
  - gu 14y ago
    You are setting up a false dichotomy here. Speaking in your analogy, the question is not between grammatical and boring and ungrammatical and exciting, but between a novel that is gripping and grammatical versus one that may be as gripping but wasn't properly edited.
- chrisaycock 14y ago
  That comment on your blog is exactly what professional programmers do in the real world: pass the test suite and move on. After all, the goal of software engineering isn't to write elegant code; it's to deliver software that solves the customer's needs. And the customer's needs are tracked via the spec, not the style guide.
  - Evbn 14y ago
    This is a short term attitude that is incompatible with building a system that grows mor ppwerful or a decade. That may be OK or maybe not, depending on your horizon and sunset plans.
    - alexkus 14y ago
      IME it's not that binary. In our corporate environment we do enough to pass the tests, with one extra 'test' being a peer review which should take into account a list of criteria that aren't easy to check for automatically; house code style, test code coverage, future maintainability, g11n/i18n-ness, etc. We often only go as far as 'just good enough' but the standard to which that is assessed is pretty high.
  - gu 14y ago
    Apart from the issue that we're talking about an introductory CS course, the greater problem remains that you'll only load up "technical debt", which may well lead to problems later on. Michael O.Church briefly talks about this in his article on startup culture: http://michaelochurch.wordpress.com/2012/07/08/dont-waste-your-time-in-crappy-startup-jobs/ http://michaelochurch.wordpress.com/2012/07/08/dont-waste-yo... Also, see the recent HN post, "Ask HN: I just inherited 700K+ lines of bad PHP. Advice?": http://news.ycombinator.com/item?id=4557919 http://news.ycombinator.com/item?id=4557919 Lastly, in my article I highlight Open Office, which still has comments in German in its source code.
alexkus 14y ago
"These test suites are often 5-10 times larger than the actual submitted code, and it is difficult to imagine anybody outside of this type of environment spending so much extra time designing (and testing!) test suites with 100% coverage." Try the (extensive!) programming challenges at http://uva.onlinejudge.org/ http://uva.onlinejudge.org/ In almost all of the challenges the example input data is sized such that even the most naive algorithm will run within a second or so. When submitted the code is judged against much larger/more-complex sets of input that will catch out inappropriate algorithm choice, unhandled edge cases, etc.
- mistercow 14y ago
  That's pretty neat. I wish that it had a little more language support though. As a recreational activity, it'd be nice to be able use something a little more modern than Java.
  - robrenaud 14y ago
    spoj.pl is very similar to acm.uva.es, but also has a much wider array of supported languages.
  - eugenejen 14y ago
    I have been pondering the possibility to build a challenge problem site that accepts solutions composed in languages other than c/c++/java/c# . Interview street has done some for languages from Haskell to scala to php. But probably I will include javascript because javascript is going to be influential on html5 apps. If anyone has similar idea and like to team up. Please contact me.
    - mistercow 14y ago
      That would be really cool. Another thing about doing it in JS: you could then potentially use something like emscripten to automatically support other languages.

by trekkin 14y ago

gu 14y ago

I wasn't talking about insufficient test cases. Your remark about extensive unit tests is therefore quite irrelevant in this context as I don't question at all that the unit tests take border cases into account. I am mostly concerned with "soft" aspects. Just consider the case where a student has to define variables, but picks variable names in a language other than English, or where control flow in a submission is more convoluted than it would have to be. Those are the cases I discuss in the article. Moments ago, someone left a very fitting comment on my blog: "I am taking the edX CS169.1 course and I find that I will consistently have a "less than elegant" solution that the auto grader accepts but that I feel is sub-par. The irony is this class has a large BDD/TDD aspect and is teaching RED-GREEN-REFACTOR, but with an auto grader once its green there is little reason to go back and refactor."

trekkin 14y ago
Let me try to make my point by a "soft" analogy, in your terminology. If a fiction book has a great, gripping plot and interesting, relatable, wonderfully done characters, then weird spelling and heavy sentences are not a big deal and can be easily fixed by a competent editor. But nothing can save a very grammatical and clean-written text that is just flat, boring, or makes no sense at all. Ask any publisher - which kind of books they prefer? Similarly in software, getting the big picture right is much much more important than "elegance" in each individual line.
- gu 14y ago
  You are setting up a false dichotomy here. Speaking in your analogy, the question is not between grammatical and boring and ungrammatical and exciting, but between a novel that is gripping and grammatical versus one that may be as gripping but wasn't properly edited.
chrisaycock 14y ago
That comment on your blog is exactly what professional programmers do in the real world: pass the test suite and move on. After all, the goal of software engineering isn't to write elegant code; it's to deliver software that solves the customer's needs. And the customer's needs are tracked via the spec, not the style guide.
- Evbn 14y ago
  This is a short term attitude that is incompatible with building a system that grows mor ppwerful or a decade. That may be OK or maybe not, depending on your horizon and sunset plans.
  - alexkus 14y ago
    IME it's not that binary. In our corporate environment we do enough to pass the tests, with one extra 'test' being a peer review which should take into account a list of criteria that aren't easy to check for automatically; house code style, test code coverage, future maintainability, g11n/i18n-ness, etc. We often only go as far as 'just good enough' but the standard to which that is assessed is pretty high.
- gu 14y ago
  Apart from the issue that we're talking about an introductory CS course, the greater problem remains that you'll only load up "technical debt", which may well lead to problems later on. Michael O.Church briefly talks about this in his article on startup culture: http://michaelochurch.wordpress.com/2012/07/08/dont-waste-your-time-in-crappy-startup-jobs/ http://michaelochurch.wordpress.com/2012/07/08/dont-waste-yo... Also, see the recent HN post, "Ask HN: I just inherited 700K+ lines of bad PHP. Advice?": http://news.ycombinator.com/item?id=4557919 http://news.ycombinator.com/item?id=4557919 Lastly, in my article I highlight Open Office, which still has comments in German in its source code.

alexkus 14y ago

"These test suites are often 5-10 times larger than the actual submitted code, and it is difficult to imagine anybody outside of this type of environment spending so much extra time designing (and testing!) test suites with 100% coverage." Try the (extensive!) programming challenges at http://uva.onlinejudge.org/ http://uva.onlinejudge.org/ In almost all of the challenges the example input data is sized such that even the most naive algorithm will run within a second or so. When submitted the code is judged against much larger/more-complex sets of input that will catch out inappropriate algorithm choice, unhandled edge cases, etc.

mistercow 14y ago
That's pretty neat. I wish that it had a little more language support though. As a recreational activity, it'd be nice to be able use something a little more modern than Java.
- robrenaud 14y ago
  spoj.pl is very similar to acm.uva.es, but also has a much wider array of supported languages.
- eugenejen 14y ago
  I have been pondering the possibility to build a challenge problem site that accepts solutions composed in languages other than c/c++/java/c# . Interview street has done some for languages from Haskell to scala to php. But probably I will include javascript because javascript is going to be influential on html5 apps. If anyone has similar idea and like to team up. Please contact me.
  - mistercow 14y ago
    That would be really cool. Another thing about doing it in JS: you could then potentially use something like emscripten to automatically support other languages.