5 ms·

I think analyzing languages through the lens of incentives ("what is this language pushing me towards/away from?") is a great idea. But if Haskell's incentives

I think analyzing languages through the lens of incentives ("what is this language pushing me towards/away from?") is a great idea. But if Haskell's incentives make it well-suited for programming in the large, why is it so rare to see large companies using Haskell? One explanation is: maybe Haskell is great for programming in the large, but is not very "accessible" to the average programmer. That is, as long as your team is competent, you can build really large, long-lived projects in Haskell more effectively than in most languages. However, if your languages requires an above-average level of competence, can you really claim that it's suited for "programming in the large?" To me, "PITL" means maintaining a huge codebase across decades, with dozens or hundreds of programmers coming and going. I'm also inclined to discount this explanation because the Haskell community often claims that Haskell isn't really that difficult to learn. Another possible explanation is that people just haven't realized that Haskell is a great PITL language yet. But Haskell has been around for quite a while now; why didn't it take this long for C++ or Java or Go or Rust to become mainstream PITL languages? I think the real reason is that Haskell actually isn't a great PITL language. Clearly it checks some of the most important boxes: type-safety, non-nullability, immutability, etc. But these are outweighed by other aspects -- most significantly, the "excessive abstraction" mentioned by the author. As admitted in the article, "abstraction is too fun!" Haskell's powerful type system, operator overloading, language extensions, etc. are all double-edged swords in this way, and in terms of PITL their costs far, far outweigh their benefits. I'd really like to see more languages that make PITL their top priority. Focusing on "safety" or "robustness" is too narrow-minded; PITL affects almost every aspect of a language: the syntax, the type system, the tooling, even the compilation speed! AFAIK, Go is the only modern language to make PITL their "north star," and it has succeeded quite well -- but there's certainly still a lot about Go that could be improved. I think we'll see more "PITL-first" languages in the near future, probably incorporating ideas from Go, Haskell, Rust, and others.

geofft 5y ago
I absolutely agree that programming in the large is about the time dimension, or, as Google puts it [1], "Software engineering is programming over time." But I do think the author recognizes that - their argument is about the fact that despite Haskell giving you less short-term productivity it also discourages worst practices that will impede future programmers. However, that still leaves us with the truth you found, that you don't see this in practice. There should really not need to be any "convincing." And it's not that people aren't allowed to use it: we all know of stories of an old application that is inexplicably business-critical, isn't in a language the company otherwise uses because someone got away writing it in a language they happened to want to try, and hasn't been rewritten, and that language is basically never Haskell. I've been pondering Google's observation and I think my conclusion is that software engineering is the art of making good APIs for future programmers. The core part is still that software engineering is the sort of programming where someone (possibly future-you) will base their work on your program, and therefore the value of the program is in how it is written and not just whether it gets today's job done. But the future programmer can always choose not to base their work on your code and do the same work you did, and the goal is that you should be making their job easier. And, in turn, the future programmer may or may not be doing software engineering, which is to say, they may just wish to consume your base code and solve that day's problem without making things better for yet another future programmer. (This is common when a project either temporarily or permanently does not expect much future development; you need to keep it running and respond to slight changes in business requirements, but it's not worth doing the engineering work to generalize your task.) In this view, it's actually important that your language facilitate both best practices and worst practices! If the language forces you to write good code all the time, then it's very hard to make things easier for future programmers, especially future programmers who are not trying to do software engineering. If it takes me five days to solve a problem well in Haskell because I can't do any worst practices, but adapting it to next month's problem also takes me five days because I still can't do any worst practices, I didn't do any software engineering - I didn't make my job any easier. One of the interesting things about Python and Ruby is that they're languages that very much optimize for worst practices (short, untyped scripts with no abstractions and dynamic variable definitions and so forth), but they also happen to give you facilities for best practices if you want. Both of them have grown very good optional type checkers in the last few years, mostly written by large tech companies that have been programming in the very large in those languages, in both time and space, and needed to deal with their code. Both of them have optional linters. Both have facilities for good abstractions like optional record types - but both also give you plain old maps. And so forth. (By contrast, consider shell, which is fantastic for programming, that is, solving immediate problems, but completely unsalvageable for software engineering, that is, building APIs for others. There is no good way to bring order to hundreds of thousands of lines of shell, even if you wanted to.) So, the Python/Ruby software engineer can build APIs for the future Python/Ruby programmer by using best practices in the long-term core they write, but the future programmer is absolutely free to use worst practices to solve their immediate problem if indeed they're solving an immediate problem. Or they can contribute to the long-term core using best practices. That seems like the happy medium here. And it's also very interesting that, as far as I know, they never set out to do this explicitly; they just happened to have just enough facilities for abstraction that over time people built more as they found themselves needing them.
- geofft 5y ago
  Forgot the reference! [1] https://abseil.io/resources/swe-book https://abseil.io/resources/swe-book Software Engineering At Google: Lessons Learned from Programming Over Time
AnimalMuppet 5y ago
A Go+Haskell=NewLang could be really interesting... or really unusable.
- zainhoda 5y ago
  There was a project that started down this road but was abandoned https://oden-lang.github.io https://oden-lang.github.io
eternityforest 5y ago
I always hear Haskell people focus on reuse in terms of mathematical abstractions, like a reusable way of composing three functions in a a pattern that's "Really everywhere if you know where to look" or something, while abstraction as OOP people understand it takes a backseat. To me, that seems hard to interpret any other way than "To use Haskell to maximum benefit you have to be very smart and able to see deeply hidden patterns with ease". As an outsider, nothing about it seems like it was designed to take 1000 random new average graduates and put them on a project and get good results. Java looks nasty and ugly but its hard to argue with how well it seems to work for large desktop apps. JS works amazingly well just because every individual project is trivial, the whole ecosystem is based on frictionless npm reuse. Haskell looks like it has potential, and studies seem to show it reduces bugs, but I have no real way to evaluate it. I don't see thousands of people making million line apps with huge teams of mediocre developers. I can't really try it myself because... I'm not about to spend 2 years working on a personal megaproject, learning the details of 2 dozen libraries,etc, just to evaluate it. So... how am I supposed to know how well Haskell does in the contexts I'm actually interested in? I could learn it, and build some toy app... but that's not going to tell me how it scales to a 10+ year monolith that is primarily about manipulating state. Rust seems like the best choice for the low level stuff these days. I don't know what the true best for high level is, but Typescript and Python+MyPy rarely disappoint.
ncmncm 5y ago
Almost every large project starts as a small project. Arguably, all of the actually-successful ones did. The only projects I have heard of that started large are rewrites of another project, in another language, that did start off small. It seems possible there are a few run under rigorous engineering management, I imagine likely only in aerospace. So, if you want a language to be good for large projects, and to get it actually ever used for them, it has to also be good for writing small projects, and good at adapting to them becoming large over time. C++ targets that role: you can write a useful 500-line program, and also maintain a million-line system, all in the same C++. Rust is making a go at that space. Common Lisp and Ada wanted a piece of that pie. Scheme, by contrast, explicitly chose not to. C is fantasmagorically unsuited for writing big systems, but is shoehorned into the role anyway. Java was imagined to be targeting the role, but its design was so crabbed as to be almost as bad at it as C; but it is gradually being grown into the role. I note in passing that PITL is unfortunately close to PITA. Maybe another term of art would be a better choice.
dnautics 5y ago
> Go is the only modern language to make PITL. I'd say elixir hits this really well. We're managing a huge project with lots of painful, legacy code that should probably be chopped off, but it's not godawful to maintain. I'm currently in a situation where we have some code hasn't been improved, so every night while europe sleeps I console into the container, hot-swap one module with some copy-pasted code, trigger a sync, and then kick the container so the sync code gets reset. Given our limited dev resources (there is a dev on staff that would be the fastest to fix it without having to learn a ton of business logic context but he's being resourced to other higher priorities now) to solve this problem, It's not the worst (this is honestly one of the least painful 'options' for managing this situation)... I mean, it will be awful if I have to do this for two months, but at that point, that's not a PL problem, that's an org problem.
foolfoolz 5y ago
this is complete elitism. languages do not “ requires an above-average level of competence” it might just have a longer learning curve. projects that are maintainable such as “huge codebase across decades, with dozens or hundreds of programmers coming and going” aren’t a special kind of program. these are just the successful ones the distinction is between “hobby/academia” and “industry”
- jcelerier 5y ago
  > languages do not “ requires an above-average level of competence” it might just have a longer learning curve. Isn't that literally the same thing ?
tome 5y ago
> why didn't it take this long for C++ or Java or Go or Rust to become mainstream PITL languages? Java and Go have had corporate behemoths sponsoring them, some of the most powerful companies the world has ever seen. Rust has also had corporate backing of a sort, but I wouldn't call it a mainstream language anyway. C++, I don't know.

geofft 5y ago

I absolutely agree that programming in the large is about the time dimension, or, as Google puts it [1], "Software engineering is programming over time." But I do think the author recognizes that - their argument is about the fact that despite Haskell giving you less short-term productivity it also discourages worst practices that will impede future programmers. However, that still leaves us with the truth you found, that you don't see this in practice. There should really not need to be any "convincing." And it's not that people aren't allowed to use it: we all know of stories of an old application that is inexplicably business-critical, isn't in a language the company otherwise uses because someone got away writing it in a language they happened to want to try, and hasn't been rewritten, and that language is basically never Haskell. I've been pondering Google's observation and I think my conclusion is that software engineering is the art of making good APIs for future programmers. The core part is still that software engineering is the sort of programming where someone (possibly future-you) will base their work on your program, and therefore the value of the program is in how it is written and not just whether it gets today's job done. But the future programmer can always choose not to base their work on your code and do the same work you did, and the goal is that you should be making their job easier. And, in turn, the future programmer may or may not be doing software engineering, which is to say, they may just wish to consume your base code and solve that day's problem without making things better for yet another future programmer. (This is common when a project either temporarily or permanently does not expect much future development; you need to keep it running and respond to slight changes in business requirements, but it's not worth doing the engineering work to generalize your task.) In this view, it's actually important that your language facilitate both best practices and worst practices! If the language forces you to write good code all the time, then it's very hard to make things easier for future programmers, especially future programmers who are not trying to do software engineering. If it takes me five days to solve a problem well in Haskell because I can't do any worst practices, but adapting it to next month's problem also takes me five days because I still can't do any worst practices, I didn't do any software engineering - I didn't make my job any easier. One of the interesting things about Python and Ruby is that they're languages that very much optimize for worst practices (short, untyped scripts with no abstractions and dynamic variable definitions and so forth), but they also happen to give you facilities for best practices if you want. Both of them have grown very good optional type checkers in the last few years, mostly written by large tech companies that have been programming in the very large in those languages, in both time and space, and needed to deal with their code. Both of them have optional linters. Both have facilities for good abstractions like optional record types - but both also give you plain old maps. And so forth. (By contrast, consider shell, which is fantastic for programming, that is, solving immediate problems, but completely unsalvageable for software engineering, that is, building APIs for others. There is no good way to bring order to hundreds of thousands of lines of shell, even if you wanted to.) So, the Python/Ruby software engineer can build APIs for the future Python/Ruby programmer by using best practices in the long-term core they write, but the future programmer is absolutely free to use worst practices to solve their immediate problem if indeed they're solving an immediate problem. Or they can contribute to the long-term core using best practices. That seems like the happy medium here. And it's also very interesting that, as far as I know, they never set out to do this explicitly; they just happened to have just enough facilities for abstraction that over time people built more as they found themselves needing them.

geofft 5y ago
Forgot the reference! [1] https://abseil.io/resources/swe-book https://abseil.io/resources/swe-book Software Engineering At Google: Lessons Learned from Programming Over Time

AnimalMuppet 5y ago

A Go+Haskell=NewLang could be really interesting... or really unusable.

zainhoda 5y ago
There was a project that started down this road but was abandoned https://oden-lang.github.io https://oden-lang.github.io

eternityforest 5y ago

I always hear Haskell people focus on reuse in terms of mathematical abstractions, like a reusable way of composing three functions in a a pattern that's "Really everywhere if you know where to look" or something, while abstraction as OOP people understand it takes a backseat. To me, that seems hard to interpret any other way than "To use Haskell to maximum benefit you have to be very smart and able to see deeply hidden patterns with ease". As an outsider, nothing about it seems like it was designed to take 1000 random new average graduates and put them on a project and get good results. Java looks nasty and ugly but its hard to argue with how well it seems to work for large desktop apps. JS works amazingly well just because every individual project is trivial, the whole ecosystem is based on frictionless npm reuse. Haskell looks like it has potential, and studies seem to show it reduces bugs, but I have no real way to evaluate it. I don't see thousands of people making million line apps with huge teams of mediocre developers. I can't really try it myself because... I'm not about to spend 2 years working on a personal megaproject, learning the details of 2 dozen libraries,etc, just to evaluate it. So... how am I supposed to know how well Haskell does in the contexts I'm actually interested in? I could learn it, and build some toy app... but that's not going to tell me how it scales to a 10+ year monolith that is primarily about manipulating state. Rust seems like the best choice for the low level stuff these days. I don't know what the true best for high level is, but Typescript and Python+MyPy rarely disappoint.

ncmncm 5y ago

Almost every large project starts as a small project. Arguably, all of the actually-successful ones did. The only projects I have heard of that started large are rewrites of another project, in another language, that did start off small. It seems possible there are a few run under rigorous engineering management, I imagine likely only in aerospace. So, if you want a language to be good for large projects, and to get it actually ever used for them, it has to also be good for writing small projects, and good at adapting to them becoming large over time. C++ targets that role: you can write a useful 500-line program, and also maintain a million-line system, all in the same C++. Rust is making a go at that space. Common Lisp and Ada wanted a piece of that pie. Scheme, by contrast, explicitly chose not to. C is fantasmagorically unsuited for writing big systems, but is shoehorned into the role anyway. Java was imagined to be targeting the role, but its design was so crabbed as to be almost as bad at it as C; but it is gradually being grown into the role. I note in passing that PITL is unfortunately close to PITA. Maybe another term of art would be a better choice.

dnautics 5y ago

> Go is the only modern language to make PITL. I'd say elixir hits this really well. We're managing a huge project with lots of painful, legacy code that should probably be chopped off, but it's not godawful to maintain. I'm currently in a situation where we have some code hasn't been improved, so every night while europe sleeps I console into the container, hot-swap one module with some copy-pasted code, trigger a sync, and then kick the container so the sync code gets reset. Given our limited dev resources (there is a dev on staff that would be the fastest to fix it without having to learn a ton of business logic context but he's being resourced to other higher priorities now) to solve this problem, It's not the worst (this is honestly one of the least painful 'options' for managing this situation)... I mean, it will be awful if I have to do this for two months, but at that point, that's not a PL problem, that's an org problem.

foolfoolz 5y ago

this is complete elitism. languages do not “ requires an above-average level of competence” it might just have a longer learning curve. projects that are maintainable such as “huge codebase across decades, with dozens or hundreds of programmers coming and going” aren’t a special kind of program. these are just the successful ones the distinction is between “hobby/academia” and “industry”

jcelerier 5y ago
> languages do not “ requires an above-average level of competence” it might just have a longer learning curve. Isn't that literally the same thing ?

tome 5y ago

> why didn't it take this long for C++ or Java or Go or Rust to become mainstream PITL languages? Java and Go have had corporate behemoths sponsoring them, some of the most powerful companies the world has ever seen. Rust has also had corporate backing of a sort, but I wouldn't call it a mainstream language anyway. C++, I don't know.