11 ms·
I think the big big result from this study is: "Python: half the size !". A dynamic language like Python is better here, 2x better. I assume similar results w
by galaxyLogic 7y ago
I think the big big result from this study is: "Python: half the size !".
A dynamic language like Python is better here, 2x better. I assume similar results would apply to other dynamic languages like JavaScript, Lisp, Smalltalk, Groovy etc.
This does not say that static typing should not be used but I think it shows unequivocally that there is a considerable extra development cost associated with static typing.
You might say that surely that additional cost would be compensated in reducing the cost of maintenance later. Maybe but I'm not sure.
Any development effort of significant size (like writing a compiler here) is a combination of writing new code and adapting and modifying code already written. "Maintenance" is part of development.
This is quite a quantitative study which gives credence to the claims of advocates of dynamic languages.
- trishume 7y agoI agree it's definitely an interesting result and a point in favour of dynamic languages. A caveat is that I'm pretty sure my friend intentionally sacrificed code quality to do it, I don't think you'd find that project as readable and understandable as the others. Another caveat is that you have to be okay with your codebase being extremely magical and metaprogramming-heavy, which many industrial users of dynamic languages avoid. As I mention, I'm personally into statically typed languages mostly for the performance and correctness benefits. I think it's plausible that on larger projects with teams the correctness benefits save enough debugging time that overall implementation time is lower, but I'm less confident in this now than I was before this comparison.
- galaxyLogic 7y agoI've read my share of cryptic JavaScript written by others and in that sense I agree that in multi-person long-term projects static typing no doubt will have its advantages. My hunch is however that what is often overlooked in development with statically typed languages is that it takes considerable time and effort to come up with the right set of types. Many examples are written showing how types almost magically make programs easier to understand. But when you read such an example what is not stated is how much effort it took to come up with just those types. One way of thinking about it is that type-definitions are really a "second program" you must write. They check upon the primary program and validate it. But that means you must write that second program as well. It's like building an unsinkable ship with two hulls one inside the other. The quality will be great but it does cost more.
- roca 7y agoNo matter what, you need a rigorous schema for your data. If you write a complex JS/Python program without doing the equivalent of "come up with the right set of types" then you will have a bad time. I'm sure in the OP here the skilled Python programmer did think carefully about the shapes of her data, she just didn't write it down. To be sure, having to write down those data structure invariants in a rigorous way that fits into the type system of your programming language has a cost. But the hard part really is coming up with the invariants, and it's dangerous to think that dynamic languages obviate the need for that.
- origin 7y agoIt's also hard to massage your invariants into a form that a type checker will accept, since you're now restricted to weird, (usually) non-Turing-complete language. A good example of this is matrix operations - there are plenty of invariants and contracts to check (e.g. multiplication must be between m x n and n x p matrices), but I don't believe there's yet a particularly convincing Haskell matrix library, in part because the range of relevant mathematical invariants don't cleanly fit into Haskell's type system. For those cases, checking the invariants at runtime is your escape hatch to utilize the full expressive power of the language.
- grumdan 7y agoThis particular example can be encoded into the Haskell type system though. For example, there's a tensor library where all operations are (according to the description) checked for the correct dimensions by the type system. It seems to require a lot of type-level magic though, and that may disqualify it for "cleanly". http://hackage.haskell.org/package/easytensor http://hackage.haskell.org/package/easytensor
- jcora 7y agoThis is an argument for more types, though, not less. Your example is trivially accomplished using dependent types.
- galaxyLogic 7y ago
- geofft 7y agoTangentially - do you know how fast her compiler was compared to the compiled-language implementations? I have a vague sense that for many applications, interpreted languages are totally fine and reaching for a compiled language is premature optimization, but I'm curious how it actually holds up for a compiler, which is more computationally heavy than your average web app or CLI.
- trishume 7y agoNo I don't know how fast her compiler was. This also isn't a good setup for comparing language performance, since all the test programs were small and there was no incentive to make your compiler fast. Many groups probably used O(n^2) algorithms in their compiler and differences in algorithm choice would probably add too much noise to get a good performance signal. That aside, speaking off the cuff totally separately from my post, I'm extremely dissatisfied with the performance of current compilers. The fastest compilers written by performance-oriented programmers can be way faster than ones you generally encounter. See luajit and Jonathan Blow's 50k+ loc/second compilers and the kind of things they do for performance. One example of a compiler task it's really difficult to do quickly in a language like Python/Ruby is non-regex parsing, I've written recursive descent parsers in Ruby and compiled languages and the Ruby ones were 1-2 orders of magnitude slower, non-JIT dynamic languages are really bad at tight loops like that.
- saagarjha 7y ago> I'm extremely dissatisfied with the performance of current compilers. The fastest compilers written by performance-oriented programmers can be way faster than ones you generally encounter. See luajit and Jonathan Blow's 50k+ loc/second compilers and the kind of things they do for performance. Lua and Jai are lot less complex than say C++: sure, LLVM isn't necessarily built to be the fastest compiler in existence, but I don't think it's fair to compare it to compilers for much simpler languages and bemoan its relative slowness.
- new4thaccount 7y agoYep, the takeaway for me was that the Python project required far less code, but we're not sure how fast it ran. Further below, the author states the inputs were so small it didn't matter. What if it did? Would the Python solution still be viable?
- bjourne 7y agoThe takeaway for me was: "The next comparison was my friend who did a compiler on her own in Python and used less than half the code we did because of the power of metaprogramming and dynamic types." So it's the output of ONE Python programmer vs teams of other languages programmers?
- galaxyLogic 7y agoThat is a good point worth emphasizing. It is well-known that teams cause overhead. Think of the Mythical Man-Moth. But in the end the other teams had 3 people that were able to maintain and develop their project further, if needed. The single-person team had only one such person.
- pm90 7y agoBut the Python code was also smaller. Easier to learn and maintain.
- blumomo 7y ago"Smaller code" does not mean easier to understand. It just means less characters to read. Maybe those characters are heavily loaded with meaning, as in the case with meta programing. You might need twice as much time to _understand_ the Python Code vs. the Rust code. The Rust code might be easier to extend, etc. So this is all comes to trade-offs at the end. All this being said, I'm still a huge Python enthusiast.
- gmueckl 7y agoSmaller code may also mean more entangled and more rigid design. But that is hard to tell from just the line count.
- earenndil 7y agoIt was also stated, though, that the reason she worked on her own was that she was a very good programmer; presumably, better than a lot of the people who worked in groups. And, as the sibling mentions, teams introduce overhead.
- mannykannot 7y agoIn my personal notion of code quality, being correct counts for a lot, and this programmer aced the tests, while apparently delivering more within the deadline that any of the multi-person teams. While writing the previous paragraph, I wrote 'programmer' where I had previously put 'implementation', because I would guess that this study's outcome is better explained by human factors than language differences. I share the attitudes you state in your last paragraph, but I would add that we should be skeptical of concepts of quality that seem plausible, but which lack empirical evidence for their effectiveness.
- danmaz74 7y agoLess code doesn't imply lower quality code. A more expressive language + less boilerplate allow you to write high quality, readable code with fewer lines.
- dmurray 7y ago> I think the big big result from this study is: "Python: half the size !". > A dynamic language like Python is better here, 2x better. I was surprised by how small the LOC benefit was from using the dynamic languages. As someone who typically reaches for Python, I'd use a statically typed language (Go or Java, most likely) much more often if I expected only twice as many lines of code. In practice I feel the same project takes 3-10 times as many LOC and that pushes it to where it is more difficult to maintain and understand.
- mpoteat 7y agoThe Python developer also added a lot of extra features. A MVP might be half again the size.
- trishume 7y agoNote my half the size estimate was after estimating the amount of code dedicated to extra features. The raw line count is only 0.7x the size of our non-test line count. Although my estimate of how many lines it would have taken without the extra features has very wide error bars.
- galaxyLogic 7y agoI have the same experience with Java. But I would have expected languages that have type-inference like Haskell, Scala and OCaml to do much better. But maybe their advanced features in fact make programs written in them harder to understand, which slows down development. Don't know.
- tigershark 7y agoI don’t think that you can consider people with 2k locs of experience in Haskell highly competent in that language.
- zmmmmm 7y agoSeems like a bit of a bold claim when the author themselves directly contradicts that in their own conclusion: > I think my overall takeaway is that design decisions make a much larger difference than the language After all, Scala was 0.7x the size and is one of the most strongly statically typed languages. So you could almost invert your conclusion and say the big result is "Python only saved 20% code lines over fully statically typed language"
- tanin 7y agoOne factor on why Python only saves 20%. Scala has a very rich standard library. Python doesn't. An example is: https://stackoverflow.com/questions/363944/python-idiom-to-return-first-item-or-none https://stackoverflow.com/questions/363944/python-idiom-to-r... Ruby would have been a better example of how succinct a dynamic-typed language can be. (I made a similar comment on the parent level.)
- saagarjha 7y ago> Scala has a very rich standard library. Python doesn't. This is not something I could have expected someone to say about Python's standard library…
- tanin 7y agoI know. It's strange to say that. But Scala's library is very very rich. Another example is that Scala offers a lot of ways to process a list like foldLeft, foldRight, unzip, headOption, lastOption, flatMap, groupBy, and many methods around Map, Set, and etc. Python probably doesn't offer many of these methods. Of course, this comes with the cost of higher learning curve.
- xenator 7y agoList of standard methods of this kind https://docs.python.org/3/library/itertools.html https://docs.python.org/3/library/itertools.html
- koala_man 7y agoGiven that they had two Rust versions where one was a third the size of the other, I don't think you should consider that a big result.
- iClaudiusX 7y agoIndeed, design choices accounted for the biggest difference by far. I like that the title frames it as a language shootout to pull people in to see if their favorite language wins (and I'm partial to Python having rewritten tens of thousands of lines of Java into numpy). Still, it would be foolish for people to come away from this brilliant analysis by ignoring the more important conclusion.
- kace91 7y agoMy experience is that you always use a type system. If the language doesn't include one, you'll end up making a bunch of extra tests and control code that in practice is constraining your data to resemble a type. It's a worse system: more verbose, harder to refactor and it's way easier to miss corner cases. It's either that or assuming that your data will contain exactly what you want to, ignoring every possibility outside the happy path, which is a recipe for disaster. That said, I have to say that modern JS produces the cleanest looking and less verbose code I've ever worked with so far. I wish there was a way to work with types without adding verbosity and harming readability.
- mannykannot 7y ago> I think the big big result from this study is: "Python: half the size!". I would agree, if and only if I thought a representative sample of Python programmers would all produce something of a similar size and just as correct, but I suspect, in this case, it's the result of one especially talented person.
- nybble41 7y agoRaw talent and prior experience no doubt had something to do with it, but there's also the fact that the Python programmer was working alone and not particularly concerned about code quality or maintainability.
- mannykannot 7y agoI agree (with the caveat that, as I wrote in another comment, I think being correct is an important aspect of quality), but this observation also argues against claims that attribute the outcome to the qualities of Python.
- massung 7y agoI enjoy hacking in dynamic languages as much as the next programmer. But, the big take-away is that "the initial implementation was done in 1/2 the code" not that the resulting code was more extensible or maintainable (by other programmers!). > You might say that surely that additional cost would be compensated in reducing the cost of maintenance later. Maybe but I'm not sure. I am sure. 100%. From many years of experience. Yes, static types come at an initial cost at initial development time. But they pay that time back in spades. This is exponentially true the larger the code base is (more to keep in one's head), the longer the project lives, and the more people are on it. Having worked on very large C/C++, Scala and Python projects, when it comes to add a major feature or perform a serious refactor, I always want the static typing and the compiler to inform me when I've missed something. Far too many times has code been checked into a large Python code base that breaks something (completely unbeknownst to the programmer), because there's a code path that's rarely executed or arguments to a function flipped, etc. That all said. There are major benefits to being able to prototype very quickly in a dynamically typed language, too.
- iLemming 7y agoThere are not too many dynamically typed languages that truly allow you not to miss static typing. One of them is Clojure. I can't explain exactly how, but somehow I think Clojure is fine without them. I don't think I can say the same thing about JS, Lua, Python or Ruby.
- yakshaving_jgt 7y agoI spent a couple of years working in a Clojure shop with people who actually like Clojure, and the experience for me was not so practically different than if everything had been written in Ruby (and indeed, half the codebase was a legacy RoR system). You either have a type-checker and compiler, or you don't.
- iLemming 7y agoWas it before Spec? In my experience - wise use of Clojure.Spec makes a big difference.
- friedman23 7y agoI don't think the reason why these languages are terser is because they are dynamic. Something that always frustrates me to no end is how many older statically types languages barely have a literal syntax.
- yxhuvud 7y agoLiteral syntax and stdlib size and APIs. The latter is especially important on iterator and container interfaces. They must fit together well and be composable. This can be exemplified using Crystal, which is close to Ruby in both terseness and APIs, but statically typed (and with static dispatch).
- tigershark 7y agoThis comparison doesn’t tell absolutely nothing about the strength and weaknesses of dynamic types languages vs static types languages in real world projects. Obviously dynamic languages are better for toy projects. If for you 4-5k loc is a project of “significant size” then we must have pretty different points of view.
- neilwilson 7y ago"This is quite a quantitative study which gives credence to the claims of advocates of dynamic languages" You have to be careful with that. In a dynamic language your test system is your compiler. It is essentially a domain specific compiler. With a statically typed language, half your tests are already written, and you just have to activate them with a type signature. For my money the trade off is down to whether forcing structures into a type system and getting the 'free advanced tests' is more advantageous than constructing a domain specific test harness 'compiler' and getting the flexibility of duck typing. And you only learn that when you run up against the limits of the type system and have to start complicating structures to work around it. In terms of a rough metric I'd suggest you have to include the test code in both cases to get a fair comparison.
- kybernetikos 7y agoI also was surprised at the magnitude of the difference and am not sure the extra lines are worth it given the strong association between number of lines and number of defects. Having said that, the largest point I took away was that the difference between languages was smaller than the difference between programmers and approaches.
- runeks 7y ago> I think the big big result from this study is: "Python: half the size !". > This does not say that static typing should not be used but I think it shows unequivocally that there is a considerable extra development cost associated with static typing. Are you saying that the bottleneck in software development is typing in text on a keyboard? If so, I thoroughly disagree. In my experience, inputting text is a small factor in development cost, with the main cost being research (figuring out what to type in), and debugging (figuring out why something you typed in doesn’t work as you thought it would).
- gameswithgo 7y agowas development time measured? loc count is not necessarily going to correlate well with time spent.
- mannykannot 7y agoThe work was done under a deadline, and some teams did not complete all of the assignment in the allotted time. The Python programmer not only completed all the required and optional features, passing 100% of all tests (both the revealed and secret ones), but "also implemented more extra features (for fun) than any other team, including an SSA intermediate representation with register allocation and other optimizations."
- p0nce 7y agoHow do we know the big result is "Python!" rather than "work alone!" or "women code better!" ? Because that was 3 features of that one sample. It's hard to make any conclusions from that article.