15 ms·

Haskell, Ada, C++: An Experiment in Prototyping Productivity (1994) [pdf]

68 points by jxub 8y ago

gpapilion 8y ago
This is a paper from 1994. C++ seems very different to me now vs the early 90s.
- mhd 8y ago
  Every language in that list is vastly different from its 90s version.
  - FatDrunknStupid 8y ago
    and really all you can conclude is that Haskell was the best language for this job at that time. I have considered this approach though. If you had two distinct dev lanes but were able to mix and match their output... I digress.
  - sacado2 8y ago
    C++ changed a lot more than Ada, though. Ada was well-designed from start.
    - mhd 8y ago
      If I remember correctly, in 1994 the STL and its associated patterns were in its utmost infancy, so compared to the old "Modern C++" or today's lambda/task/auto C++, any language will be strong & stable. Still, there have been some changes since Ada83...
- Accipitriform 8y ago
  > C++ seems very different to me now vs the early 90s. True, but it's still very far from optimal. For something MUCH closer to optimal, see Rust. All that said, from a practical standpoint I think Julia is near perfect as both a prototyping and implementation language, with the ability to drop to Rust/C/assembly easily as needed. High productivity, and C-like performance. That's a sweet spot.
  - reddit_clone 8y ago
    Is Julia used for general purpose programming? I hear of it only in the context of Data Science.
    - byt143 8y ago
      Yes. see this as an example: http://genieframework.com/ http://genieframework.com/
    - timClicks 8y ago
      Julia attracts people that are interested in data science, but it's also a very capable language outside of that field.
    - jonathanstrange 8y ago
      I'm generally interested in programming languages and have taken a look at hundreds, if not thousands in the past decades. Based on that experience, I'd say that almost no language is a general purpose programming language, because that would include cross-platform GUI programming. Except for C, C++, and Python, every binding to GUI toolkits I've seen was incomplete, buggy and not fully tested, not ready for prime time, eternally promising without ever getting there, lagging behind the toolkit versions, hard to setup and compile, and so forth. There is always something about the glue code, versioning, or portability that gets in your way. Typical example: I've decided to make a little GUI application in gtk3 for Go and encountered the first show-stopper already after two weeks. Yes, in theory this can be fixed by adding bindings yourself and submitting pull requests, but once you start with this it kind of defeats the purpose of using another language in the first place. (And there is a second issue of this kind already waiting in line...) The same is true for Julia, which I have evaluated for this purpose, too. The GUI bindings are not yet complete, and experience has told me that if bindings are not yet complete, there is a high chance they will never be complete. Once they are completed, they are already outdated. So if you include GUI applications into "general purpose", the answer is practically always "No". If you exclude GUI applications, on the other hand, then almost every programming language on earth is sufficiently general purpose.
    - 8y ago
- deleted 8y ago
  [deleted]
NikkiA 8y ago
Interesting that the lisp prototype took the least amount of time to develop, yet was considered less understable code than the haskell prototype(s).
- tmm84 8y ago
  If it used macros I bet that had some effect on the ranking. Macros and CLOS are two things that really take some time to sink in and understand for programmers that don't use lisp much.
  - sshine 8y ago
    I bet it did. In the extreme case, using macros makes every program a slightly different DSL.
    - lispm 8y ago
      I don't think they wrote any macros. The language they used is Common Lisp with a special library for programming with relations on top. So they used a DSL library for relational programming. Something like a Lisp with a kind of in-core database with a rich language for it. That kind of programming is extremely rare.
- sevensor 8y ago
  I thought it was interesting how quickly the lisp result was dismissed. I'm not particlularly partisan one way or the other, and it's to the authors' credit to have included it at all. However, this is a paper with "foregone conclusion" written all over it.
pishpash 8y ago
Trying to find the most productive language for all tasks seems like a quixotic quest. Each language encodes a set of expressions that are succinct to communicate, at the expense of some others. There are some basic information theoretical and computational complexity reasons for this. There will always be different languages that are best suited to certain tasks, and the goal should be to make it easy (tools, systems) to work with many languages equally well, or to allow switching anywhere like with human languages.
flukus 8y ago
I love awk within certain domains, but the the overlap between where awk is suitable and something big and complex enough to serve as a good example of programmer productivity is essentially nil. There are some gems in the results section though: > It is interesting to note that the developer originally reported only 101 lines of code, but fully admitted attempting to pack as many statements into each 80-column row as possible! The line count of 250 is based on looking at the actual code in [LBK+94] and estimating a line count based on a more “reasonable” notion of program formatting.
ajxs 8y ago
I believe it was Dr Robert Dewar of AdaCore fame who I first heard remark that ( paraphrased ) every time a bug gets missed at a stage of development ( compile-time to alpha, to beta, to production, etc ) the cost of it's resolution increases by an order of magnitude. I'm certainly by no means dismissing the value of concise languages for expressing ideas fluently, but brevity and expressiveness are only two factors influencing the value of a programming language in a specific domain. It's definitely an interesting paper, every time I see it I can't help but think that it's a little disingenuous to conduct an apples to oranges comparison of languages on these particular merits. It's much more difficult to weigh up the relative investment of time necessary to write code that is equally robust, maintainable and fit for purpose. Here's the video I got the Dr Dewar quote from: https://www.youtube.com/watch?v=0yXwnk8Cr0c https://www.youtube.com/watch?v=0yXwnk8Cr0c
- flukus 8y ago
  As far as costs go, 90% of it is in the "maintenance" phase from and usually be other developers, something this didn't address. Brevity certainly helps there but only up to a point, usually the most concise languages are the dynamic ones which can be much harder to maintain.
- fulafel 8y ago
  This sounds like the oft parroted but rarely fully sourced quote that turns out to be from some 60s IBM journal, with questionable applicability to the modern age. see eg https://news.ycombinator.com/item?id=15216407 https://news.ycombinator.com/item?id=15216407
  - ajxs 8y ago
    I didn't really look at the quote as being a literal, canonical truth. I have my own doubts about it being _literally_ true, but I feel that the underlying idea that "the longer a bug persists through the development process the costlier it becomes to fix" is true.
  - nickpsecurity 8y ago
    Most evidence I've seen supports it. This is especially true when people start integrating stuff with or building on top of a deliverable. Even more if that stuff was designed with the deliverables good and bad behavior in mind. It becomes a legacy system that sticks around. Knocking out its problems ahead of time can have considerable savings. Plus, a benefit not often mentioned is that knocking problems out at design time is easier since everything is fresh in developers' minds. They just wrote it. Now, it might not apply in an environment where everything is designed to be pluggable and easy to change without breaks. Maybe folks should run some new studies on such projects to see if there's any major benefits for up-front investments in quality. Enough to pay off that up-front investment like in the other projects.
yakshaving_jgt 8y ago
What this tells me most of all, is that humans will only acknowledge the legitimacy of studies that support their predetermined opinions. The arguments for and against a type system like Haskell's have been going on seemingly forever, and the commonly parroted line from e.g., Clojure or JavaScript enthusiasts (like the curly-haired San Francisco guy) is that there has never been a single study to support the idea that a type system like Haskell's is generally a cheaper way to write reasonably robust software. Point them to a study like this, and you're met with "Nuh-uh! Not valid! Nope!" My observation is supported by section 7, "Lessons Learned". > …The reaction from the other participants, however, in particular those not familiar with the advantages of functional programming, was somewhat surprising, and is worth some discussion.… > …In conducting the independent design review at Intermetrics, there was a significance sense of disbelief. > …some observers have simply discounted the results because in their minds the use of higher-order functions to capture regions was just a trick that would probably not be useful in other contexts. One observer described the solution as “cute but not extensible” > We mention these responses because they must be anticipated in the future. If functional languages are to become more widely used, various sociological and psychological barriers must be overcome. As a community we should be aware of these barriers and realize that they will not disappear overnight. They're right. These barriers haven't disappeared overnight. They haven't even disappeared after a quarter of a century. People still come up with stupid reasons why they don't like Haskell.
- fnord123 8y ago
  >They're right. These barriers haven't disappeared overnight. They haven't even disappeared after a quarter of a century. The barriers disappeared since functional concepts were absorbed into other languages. Almost all languages have higher order functions. Many languages have a preference for immutable data. Recursion was rightly not adopted as iterators are a better solution - keep the map/filter/fold, leave the TCO. >People still come up with stupid reasons why they don't like Haskell One expects that if Haskell was superior, then it would be impossible for successful Haskell projects to materialize as they've done with Python, Go, Rust, Java, C++. And yet, successful Haskell projects are rare. There must be a because there is a hidden variable that this study in 1994 didn't capture. There's certainly not a cabal of project managers forcing C++ down peoples' throats. So instead of calling the reasons stupid, why not try to find out what this hidden variable is?
ff_ 8y ago
This is an interesting paper, and I'd like a replication today, 25y after: I believe Haskell has matured a lot in all of this time. As an experiment, some months ago we prototyped a new service in Haskell at work. We timeboxed a week for it (two people, proficient in Haskell), and the result is that (1) the Product Owners were really pleased with it, and (2) even if it was really bad Haskell code, we _still_ managed to encode business logic in the types so that it would always hold some important invariants. As a result we had to fix a surprisingly small amount of things before putting it in production. The best thing is that as it happens we couldn't properly rewrite the prototype, so today's service is still the same codebase, BUT it's not a Python blob and it was such a breeze to refactor. EDIT: would anyone be interested in a blog post "Rapid Prototyping in Haskell"?
- smu 8y ago
  Yes, I would be very interested!
  - WMCRUN 8y ago
    Same
- deckiedan 8y ago
  "Rapid Prototyping in Haskell" would be fantastic.
- f0rgot 8y ago
  Yes please!
- zephyrfalcon 8y ago
  What is a "Python blob"?
  - sevensor 8y ago
    As someone who writes a lot of Python, I'd say a Python blob is a half-formed idea expressed in Python without any particular thought given to design or conceptual clarity. Just enough to get the job done if you don't encounter any surprises, but not something you'd want to maintain. You expect it not to handle all of the edge cases and when it raises an exception you jump in and code around it at the point where it's raised, rather than fixing the root cause, which is lack of design. Unfortunately, a lot of Python starts out like this and then ends up being maintained anyway. edit: What makes it blobbly is its lack of structure. It doesn't express any clear idea of the domain and tends to devolve into spaghetti.
otabdeveloper1 8y ago
> Haskell and C++ from 1994 What is this, the Guild of Legacy Crap Archeology?
egberts 8y ago
Julia is also getting there the fastest, IMHO.
axilmar 8y ago
I do not see anything in the problem description or in the analysis of the Haskell advantages that couldn't be handled by c++ or coded in c++ in a fairly similar manner to Haskell so as that to have similar advantages.
- vnorilo 8y ago
  C++ in 1994 was quite a different animal.
platz 8y ago
As a Haskeller, I am extremely wary of seeing this study linked to over and over again.

68 points by jxub 8y ago

gpapilion 8y ago

This is a paper from 1994. C++ seems very different to me now vs the early 90s.

mhd 8y ago
Every language in that list is vastly different from its 90s version.
- FatDrunknStupid 8y ago
  and really all you can conclude is that Haskell was the best language for this job at that time. I have considered this approach though. If you had two distinct dev lanes but were able to mix and match their output... I digress.
- sacado2 8y ago
  C++ changed a lot more than Ada, though. Ada was well-designed from start.
  - mhd 8y ago
    If I remember correctly, in 1994 the STL and its associated patterns were in its utmost infancy, so compared to the old "Modern C++" or today's lambda/task/auto C++, any language will be strong & stable. Still, there have been some changes since Ada83...
Accipitriform 8y ago
> C++ seems very different to me now vs the early 90s. True, but it's still very far from optimal. For something MUCH closer to optimal, see Rust. All that said, from a practical standpoint I think Julia is near perfect as both a prototyping and implementation language, with the ability to drop to Rust/C/assembly easily as needed. High productivity, and C-like performance. That's a sweet spot.
- reddit_clone 8y ago
  Is Julia used for general purpose programming? I hear of it only in the context of Data Science.
  - byt143 8y ago
    Yes. see this as an example: http://genieframework.com/ http://genieframework.com/
  - timClicks 8y ago
    Julia attracts people that are interested in data science, but it's also a very capable language outside of that field.
  - jonathanstrange 8y ago
    I'm generally interested in programming languages and have taken a look at hundreds, if not thousands in the past decades. Based on that experience, I'd say that almost no language is a general purpose programming language, because that would include cross-platform GUI programming. Except for C, C++, and Python, every binding to GUI toolkits I've seen was incomplete, buggy and not fully tested, not ready for prime time, eternally promising without ever getting there, lagging behind the toolkit versions, hard to setup and compile, and so forth. There is always something about the glue code, versioning, or portability that gets in your way. Typical example: I've decided to make a little GUI application in gtk3 for Go and encountered the first show-stopper already after two weeks. Yes, in theory this can be fixed by adding bindings yourself and submitting pull requests, but once you start with this it kind of defeats the purpose of using another language in the first place. (And there is a second issue of this kind already waiting in line...) The same is true for Julia, which I have evaluated for this purpose, too. The GUI bindings are not yet complete, and experience has told me that if bindings are not yet complete, there is a high chance they will never be complete. Once they are completed, they are already outdated. So if you include GUI applications into "general purpose", the answer is practically always "No". If you exclude GUI applications, on the other hand, then almost every programming language on earth is sufficiently general purpose.
  - 8y ago
deleted 8y ago
[deleted]

NikkiA 8y ago

Interesting that the lisp prototype took the least amount of time to develop, yet was considered less understable code than the haskell prototype(s).

tmm84 8y ago
If it used macros I bet that had some effect on the ranking. Macros and CLOS are two things that really take some time to sink in and understand for programmers that don't use lisp much.
- sshine 8y ago
  I bet it did. In the extreme case, using macros makes every program a slightly different DSL.
  - lispm 8y ago
    I don't think they wrote any macros. The language they used is Common Lisp with a special library for programming with relations on top. So they used a DSL library for relational programming. Something like a Lisp with a kind of in-core database with a rich language for it. That kind of programming is extremely rare.
sevensor 8y ago
I thought it was interesting how quickly the lisp result was dismissed. I'm not particlularly partisan one way or the other, and it's to the authors' credit to have included it at all. However, this is a paper with "foregone conclusion" written all over it.

pishpash 8y ago

Trying to find the most productive language for all tasks seems like a quixotic quest. Each language encodes a set of expressions that are succinct to communicate, at the expense of some others. There are some basic information theoretical and computational complexity reasons for this. There will always be different languages that are best suited to certain tasks, and the goal should be to make it easy (tools, systems) to work with many languages equally well, or to allow switching anywhere like with human languages.

flukus 8y ago

I love awk within certain domains, but the the overlap between where awk is suitable and something big and complex enough to serve as a good example of programmer productivity is essentially nil. There are some gems in the results section though: > It is interesting to note that the developer originally reported only 101 lines of code, but fully admitted attempting to pack as many statements into each 80-column row as possible! The line count of 250 is based on looking at the actual code in [LBK+94] and estimating a line count based on a more “reasonable” notion of program formatting.

ajxs 8y ago

I believe it was Dr Robert Dewar of AdaCore fame who I first heard remark that ( paraphrased ) every time a bug gets missed at a stage of development ( compile-time to alpha, to beta, to production, etc ) the cost of it's resolution increases by an order of magnitude. I'm certainly by no means dismissing the value of concise languages for expressing ideas fluently, but brevity and expressiveness are only two factors influencing the value of a programming language in a specific domain. It's definitely an interesting paper, every time I see it I can't help but think that it's a little disingenuous to conduct an apples to oranges comparison of languages on these particular merits. It's much more difficult to weigh up the relative investment of time necessary to write code that is equally robust, maintainable and fit for purpose. Here's the video I got the Dr Dewar quote from: https://www.youtube.com/watch?v=0yXwnk8Cr0c https://www.youtube.com/watch?v=0yXwnk8Cr0c

flukus 8y ago
As far as costs go, 90% of it is in the "maintenance" phase from and usually be other developers, something this didn't address. Brevity certainly helps there but only up to a point, usually the most concise languages are the dynamic ones which can be much harder to maintain.
fulafel 8y ago
This sounds like the oft parroted but rarely fully sourced quote that turns out to be from some 60s IBM journal, with questionable applicability to the modern age. see eg https://news.ycombinator.com/item?id=15216407 https://news.ycombinator.com/item?id=15216407
- ajxs 8y ago
  I didn't really look at the quote as being a literal, canonical truth. I have my own doubts about it being _literally_ true, but I feel that the underlying idea that "the longer a bug persists through the development process the costlier it becomes to fix" is true.
- nickpsecurity 8y ago
  Most evidence I've seen supports it. This is especially true when people start integrating stuff with or building on top of a deliverable. Even more if that stuff was designed with the deliverables good and bad behavior in mind. It becomes a legacy system that sticks around. Knocking out its problems ahead of time can have considerable savings. Plus, a benefit not often mentioned is that knocking problems out at design time is easier since everything is fresh in developers' minds. They just wrote it. Now, it might not apply in an environment where everything is designed to be pluggable and easy to change without breaks. Maybe folks should run some new studies on such projects to see if there's any major benefits for up-front investments in quality. Enough to pay off that up-front investment like in the other projects.

yakshaving_jgt 8y ago

What this tells me most of all, is that humans will only acknowledge the legitimacy of studies that support their predetermined opinions. The arguments for and against a type system like Haskell's have been going on seemingly forever, and the commonly parroted line from e.g., Clojure or JavaScript enthusiasts (like the curly-haired San Francisco guy) is that there has never been a single study to support the idea that a type system like Haskell's is generally a cheaper way to write reasonably robust software. Point them to a study like this, and you're met with "Nuh-uh! Not valid! Nope!" My observation is supported by section 7, "Lessons Learned". > …The reaction from the other participants, however, in particular those not familiar with the advantages of functional programming, was somewhat surprising, and is worth some discussion.… > …In conducting the independent design review at Intermetrics, there was a significance sense of disbelief. > …some observers have simply discounted the results because in their minds the use of higher-order functions to capture regions was just a trick that would probably not be useful in other contexts. One observer described the solution as “cute but not extensible” > We mention these responses because they must be anticipated in the future. If functional languages are to become more widely used, various sociological and psychological barriers must be overcome. As a community we should be aware of these barriers and realize that they will not disappear overnight. They're right. These barriers haven't disappeared overnight. They haven't even disappeared after a quarter of a century. People still come up with stupid reasons why they don't like Haskell.

fnord123 8y ago
>They're right. These barriers haven't disappeared overnight. They haven't even disappeared after a quarter of a century. The barriers disappeared since functional concepts were absorbed into other languages. Almost all languages have higher order functions. Many languages have a preference for immutable data. Recursion was rightly not adopted as iterators are a better solution - keep the map/filter/fold, leave the TCO. >People still come up with stupid reasons why they don't like Haskell One expects that if Haskell was superior, then it would be impossible for successful Haskell projects to materialize as they've done with Python, Go, Rust, Java, C++. And yet, successful Haskell projects are rare. There must be a because there is a hidden variable that this study in 1994 didn't capture. There's certainly not a cabal of project managers forcing C++ down peoples' throats. So instead of calling the reasons stupid, why not try to find out what this hidden variable is?

ff_ 8y ago

This is an interesting paper, and I'd like a replication today, 25y after: I believe Haskell has matured a lot in all of this time. As an experiment, some months ago we prototyped a new service in Haskell at work. We timeboxed a week for it (two people, proficient in Haskell), and the result is that (1) the Product Owners were really pleased with it, and (2) even if it was really bad Haskell code, we _still_ managed to encode business logic in the types so that it would always hold some important invariants. As a result we had to fix a surprisingly small amount of things before putting it in production. The best thing is that as it happens we couldn't properly rewrite the prototype, so today's service is still the same codebase, BUT it's not a Python blob and it was such a breeze to refactor. EDIT: would anyone be interested in a blog post "Rapid Prototyping in Haskell"?

smu 8y ago
Yes, I would be very interested!
- WMCRUN 8y ago
  Same
deckiedan 8y ago
"Rapid Prototyping in Haskell" would be fantastic.
f0rgot 8y ago
Yes please!
zephyrfalcon 8y ago
What is a "Python blob"?
- sevensor 8y ago
  As someone who writes a lot of Python, I'd say a Python blob is a half-formed idea expressed in Python without any particular thought given to design or conceptual clarity. Just enough to get the job done if you don't encounter any surprises, but not something you'd want to maintain. You expect it not to handle all of the edge cases and when it raises an exception you jump in and code around it at the point where it's raised, rather than fixing the root cause, which is lack of design. Unfortunately, a lot of Python starts out like this and then ends up being maintained anyway. edit: What makes it blobbly is its lack of structure. It doesn't express any clear idea of the domain and tends to devolve into spaghetti.

otabdeveloper1 8y ago

> Haskell and C++ from 1994 What is this, the Guild of Legacy Crap Archeology?

egberts 8y ago

Julia is also getting there the fastest, IMHO.

axilmar 8y ago

I do not see anything in the problem description or in the analysis of the Haskell advantages that couldn't be handled by c++ or coded in c++ in a fairly similar manner to Haskell so as that to have similar advantages.

vnorilo 8y ago
C++ in 1994 was quite a different animal.

platz 8y ago

As a Haskeller, I am extremely wary of seeing this study linked to over and over again.