5 ms·
One problem with this talk (and many other talks about static typing) is that it seems to take at faith value the common claim that static typing is a net benef
by rbehrends 12y ago
One problem with this talk (and many other talks about static typing) is that it seems to take at faith value the common claim that static typing is a net benefit for program correctness because it prevents programs from compiling that would otherwise create runtime errors (e.g., at around the 13 minute mark and again after the 18 minute mark). [1]
This seems like it's just common sense, but common sense claims do not always hold up in science and (as with many other claims regarding software engineering benefits that presumably derive from this or that language feature), it has rarely been tested.
This is not surprising: most computer science departments don't have the infrastructure to do work that involves human test subjects (unlike, say, psychology departments). Not only is there an ethics review board that has to sign off on such tests (no matter how harmless they may seem), creating meaningful tests involving non-trivial programming tasks can be pretty involved (a talk by the author of the first paper I discuss below was that one problem with the study was that it had been very expensive).
But at least some such tests have been done, and the conclusions are not necessarily what common sense would tell us.
Consider for example, the following series of papers:
Hanenberg, S. An Experiment About Static and Dynamic Type Systems - Doubts About the Positive Impact of Static Type Systems on Development Time. In Proceedings of OOPSLA/SPLASH 2010. ACM, Reno 2010. [2]
From the conclusion: "In this paper we presented results of an experiment that explores the impact of static and dynamic type systems on the development of a piece of software (a parser) with 49 subjects. We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting test cases fulfilled by the parser. In non of these measured points the use of the static type system turned out to have a significant positive impact. In the first case, the use of the statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured."
Now, that is one data point that only relates to the initial development of a piece of software, too (and as we'll see in a moment, it's not as though dynamic typing is a free lunch, once we look at maintenance tasks), but it does contradict common sense: The defect rate of the statically typed software is not actually better (at least not in a statistically significant way). Earlier studies had also shown that statically typed languages (unsurprisingly) did have benefits over languages that performed neither compile-time nor runtime type checks (such as ANSI C vs. K&R C w.r.t. function arguments).
Now, a follow-up paper:
Kleinschmager, S.; Hanenberg, S.; Robbes, R.; Tanter, É. & Stefik, A. Do static type systems improve the maintainability of software systems? An empirical study. IEEE 20th International Conference on Program Comprehension, ICPC 2012, Passau, Germany 2012, S. 153-162. [3]
Three conclusions from this paper on the effect of static type systems on software maintenance: (1) "Static type systems help humans use a new set of classes." (2) "Static type systems make it easier for humans to fix type errors." (3) "For fixing semantic errors, [...] no differences with respect to human development times [was observed]."
The paper also references a number of other studies with related results that may be interesting.
A second follow-up paper was the following:
Spiza, S. & Hanenberg, S. Type Names without Static Type Checking already Improve the Usability of APIs (As Long as the Type Names are Correct) - An Empirical Study. Proceedings of the ACM Conference on Aspect-Oriented Software Development, AOSD '14, Lugano, Swizerland 2014. [4]
From the abstract: "This paper describes an experiment with 20 participants that has been performed in order to check whether developers using an unknown API already benefit (in terms of development time) from the pure syntactical representation of type names without static type checking. The result of the study is that developers do benefit from the type names in an API’s source code. But already a single wrong type name has a measurable significant negative impact on the development time in comparison to APIs without type names."
What we're seeing is that while there do seem to be measurable benefits from static typing, the actual nature of these benefits may not be what one may expect on the basis of common sense alone (it's actually even more complicated than I can convey in this brief summary).
[1] To be clear, the talk makes many other immensely sensible points, and I don't think there's even enough material to conclusively say this one is right or wrong; my point is that it's essentially unproven either way, and that there is evidence that our intuition may be wrong here.
[2] https://courses.cs.washington.edu/courses/cse590n/10au/hanenberg-oopsla2010.pdf https://courses.cs.washington.edu/courses/cse590n/10au/hanen...
[3] http://pleiad.dcc.uchile.cl/papers/2012/kleinschmagerAl-icpc2012.pdf http://pleiad.dcc.uchile.cl/papers/2012/kleinschmagerAl-icpc...
[4] http://dl.acm.org/citation.cfm?id=2577098 http://dl.acm.org/citation.cfm?id=2577098
- zwieback 12y agoThanks so much for this point. I think the key here is conclusion (3) about semantic errors. I remember endless discussions about the Liskov Substitution Principle and how it relates to subtyping/subclassing. I think what people easily forget that static types do almost nothing for semantics and the notion of "behavioral subtyping" is a lot more interesting but also a lot more elusive. And I say all this as a lifelong static typer, by the way, and not even of the fancy Haskell variety. I know what not to expect from static type checking so I'm a lot happier with what it does provide me.
- deleted 12y ago[deleted]
- bkirwi 12y agoJust took a quick look at paper #2, which is quite fascinating -- to try and isolate the effects of static / dynamic typing, they developed two slightly different programming languages which varied along only this axis. I have some sympathy with this approach, but I'm not convinced you can actually separate things in this way. The type systems I actually find useful are not just a static analysis layer, but have significant impacts on the semantics of the language and the way libraries are designed. I'm not sure you could just 'turn off' the type system in Haskell and have anything even remotely sensible, for example. I sometimes wonder if there's enough mature open-source projects in the wild now to be able to draw some statistical conclusions about the long-term effects of language choice, but that would be quite a project...
- StefanKarpinski 12y ago> I'm not sure you could just 'turn off' the type system in Haskell and have anything even remotely sensible, for example. You can actually do exactly this: https://ghc.haskell.org/trac/ghc/wiki/DeferErrorsToRuntime https://ghc.haskell.org/trac/ghc/wiki/DeferErrorsToRuntime Of course, the type system doesn't vanish but checking types is deferred until runtime, turing Haskell into a dynamic language with an unusually elaborate tag system.
- 12y ago