17 ms·

The Costs of Programming Language Fragmentation

137 points by jp_sc 8y ago

DarkWiiPlayer 8y ago
That reflects rather well how I feel about the recent DSL craze, specially in the ruby community. It's also how I feel about libraries and frameworks. Programming languages probably suffer the least from this phenomenon due to how difficult it is to create a complete language, with compilers and all. In contrast, languages that just extend others often do benefit from existing communities. Take moonscript¹ for example, it's just a new syntax for Lua, so you can use all the existing libraries at no cost. Or take Terra², which can make use of all the C libraries out there and the Lua libraries at the same time. ¹ http://moonscript.org/ http://moonscript.org/ ² http://terralang.org/ http://terralang.org/
- flukus 8y ago
  Is the DSL craze really new? Look at a lot of the early history of unix and you'll see sh, sed, awk, tex and probably a heap of others that didn't survive. Not to mentions lex/yacc, two DSL's to make DSLs. The idea of specialized languages for specialized tasks has been around for decades.
- smadge 8y ago
  An example that comes up often for me are EDSLs for database queries. I do see the value of them, but on the other hand it causes you to move away from a “lingua Franca” of database querying to a programming environment specific one. What probably makes sense is 1) evolving SQL or SQL tooling to support more features that people think they need (an example that comes to mind is type safe queries) and 2) better support for binding these queries to different languages.
  - lmm 8y ago
    External DSLs (even with "cross-language bindings") are much worse to work with than than internal ones and SQL is no exception. IMO what's really needed is a willingness for databases to step away from SQL and expose an API that's more friendly to modern programming languages (or better yet, make the database embeddable as a library rather than a framework you have to build your application into). The eDSLs help a little, but as long as they're obliged to compile into SQL strings there's a limit to how effective they can be.
- vidarh 8y ago
  Most of "the DSL craze" in Ruby is just defining methods. It's rarely new languages, but simply a matter of taking full advantage of the language to let you talk about your domain in a more natural way. It's not so much a "dsl craze" in Ruby as writing idiomatic Ruby well, the same was as e.g. Smalltalk or lisps also tend to make you create your own vocabulary that is close to indistinguishable (more so than in Ruby, in fact, where the block syntax and control structure syntax differ) from the language itself.
- lassejansen 8y ago
  Not sure if "recent" is the correct attribute for DSLs in Ruby. https://www.infoq.com/news/2007/06/dsl-or-not https://www.infoq.com/news/2007/06/dsl-or-not
michaelmrose 8y ago
"However, I hope people consider carefully the social costs of creating a new programming language especially if it becomes popular, and understand that in some cases creating a popular new language could actually be irresponsible." Passion isn't fungible and there is zero gauruntee that the person who creates an interesting new language wouldn't have chosen to binge watch Netflix instead. Furthermore without a crystal ball it's difficult to separate ahead of time which efforts will move us forward in some small or large ways. The entire premise of this post is flawed.
- jacquesm 8y ago
  > it's difficult to separate ahead of time which efforts will move us forward in some small or large ways. That's the point: most efforts do not move us forward but instead move us backwards because fewer people are working on the things that matter. It's a solid argument and it applies to far more of the open source community than just to programming languages, in fact programming languages are the smaller part of the issue, but the argument still applies. > The entire premise of this post is flawed. I don't think so. I would not dismiss an article without at least trying to make a genuine effort to understand the point an article makes. The fact is that it takes effort to launch a new programming language beyond just writing some code and the long term commitment should be there if you are going to let other people run their production systems on what you throw into the world. This doesn't mean you don't get to scratch your itch, it means that once your programming language gains adoption beyond some people playing around with it you can't just walk off and say that it isn't your problem. Oh, and it is 'guarantee'.
  - a_imho 8y ago
    The fact is that it takes effort to launch a new programming language beyond just writing some code and the long term commitment should be there if you are going to let other people run their production systems on what you throw into the world. From my admittedly limited knowledge JavaScript seems to contradict this claim. I don't agree with the article either, one could make the same argument for every piece of new code written.
alexmorley 8y ago
I understand the concern but it assumes that open-source contributions are a zero-sum game, which I don't think they are. New languages often encourage users to become contributors where they might have remained users in the old language [citation needed]
ChrisCinelli 8y ago
It is not just creating a new language. In a company environment you see a huge cost when people want to start using a new language or/and framework. It comes down to values and what push people forward. I am not saying one language should be enough. I am saying that for example that even for a company that already has a library of more than a few components built in React a a medium codebase using it, starting using Vue will add a quite large cost. Even if a framework is 20% better in some metrics, having to rewrite some reusable components (assuming they were written well) in another framework is going to sunk a bunch of time. There is a learning curve that people need to go through. Hardly the first time you use a framework, you end up writing the most awesome code in that framework. There are bugs, there may not be a lot of documentation or open source libraries. From an engineering prospective is going to be fun and you are going to learn something new but from a business prospective it is rarely a good decision. Same thing is probably true when teams in company with a large Java codebase want to switch to Scala. On the other end when you have a 10x better framework or technology you are better paying attention and make sure you plan a transition sooner than later. For example, I think GraphQl is a order of magnitude better than REST when you consider a whole end-to-end system. Engineers, especially the smart ones, get bored quickly. Inventing a new largely adopted language or framework is an irresistible calling from inside. Exploring new territories is also another source of growth. It is all about values and the way we go about satisfying them.
TheAceOfHearts 8y ago
I think this could be mitigated in some cases with better tooling. Many programming languages just make it needlessly difficult to interact with anything that's not part of their ecosystem. Check out parcel [0], a web application bundler. It has built-in support for lots of different assets [1], with two notable inclusions being ReasonML and Rust! In this blog post [2] they highlight how easy it is to import Rust code from JavaScript. Another neat example is Objective-C bridging on macOS [3]. The code usually doesn't end up looking very pretty, and it can be brittle at times, but with JavaScriptObjC you can interact directly with all the native APIs using JavaScript. Here's a blog post [4] showing how to write a native app on macOS using JavaScriptObjC. [0] https://github.com/parcel-bundler/parcel https://github.com/parcel-bundler/parcel [1] https://parceljs.org/assets.html https://parceljs.org/assets.html [2] https://medium.com/@devongovett/parcel-v1-5-0-released-source-maps-webassembly-rust-and-more-3a6385e43b95 https://medium.com/@devongovett/parcel-v1-5-0-released-sourc... [3] https://developer.apple.com/library/archive/documentation/LanguagesUtilities/Conceptual/MacAutomationScriptingGuide/HowMacScriptingWorks.html#//apple_ref/doc/uid/TP40016239-CH73-SW4 https://developer.apple.com/library/archive/documentation/La... [4] https://tylergaw.com/articles/building-osx-apps-with-js/ https://tylergaw.com/articles/building-osx-apps-with-js/
reacweb 8y ago
IMHO, the author is completely wrong and the sentence "C is the desert island language" is closer to reality (see http://www-cs-students.stanford.edu/~blynn/c/intro.html http://www-cs-students.stanford.edu/~blynn/c/intro.html). The fact that such a poor language remains the single sane choice to build the kernel of Linux is a proof of the lack of languages. C++ was already a mess in 98. Python started as simple but has added more and more complex syntaxes. Java is in the hands of Satan, Ada is too verbose, nobody takes the effort to learn it, Haskell and SCALA are niche languages not so easy to learn... IMHO, there is a need for a polyvalent language that would enable development of Linux kernel (low level and performance), that would be easy to learn and that would enable quick development (dynamic typing, syntactic sugar, ...).
- jacquesm 8y ago
  > there is a need for a polyvalent language that would enable development of Linux kernel (low level and performance), that would be easy to learn and that would enable quick development That's a pretty tall order for a single language to support all those desires. But Rust seems to at least aim for that space (it doesn't check all your boxes though).
- pjc50 8y ago
  Dynamic typing and garbage collection go directly against the "low level and performant" goals. You can avoid GC with clever lifetime rules (Rust) but that loses "easy to learn".
- nineteen999 8y ago
  I'm not sure how old that article is, but parts of gcc suite have been rewritten in a subset of C++ for several years now (https://lwn.net/Articles/542457/ https://lwn.net/Articles/542457/), starting from over 10 years ago.
  - reacweb 8y ago
    Yes, I know that. C++ is also very much used for game development. C++ allows to build very impressive abstractions that are very handy. The main drawback is that this flexibility has a huge impact on the language. When you switch between two projects writen in C++, it is almost as if they were using a different language. I have understood that this language was a mess in 98 when I have bought the standard and read it completely. In professional projects, I always try to avoid it in favor of java despite all the years I have spent learning and using C++.
chubot 8y ago
It's hard to do, but I would like to see more overhauls of existing languages. I wrote a relevant comment about the approach of Checked C vs. Rust here: https://news.ycombinator.com/item?id=17944310 https://news.ycombinator.com/item?id=17944310 (tl;dr Software gets rewritten much less frequently than you think. Rust is going to add to the existing ecosystem in C++ and C, not replace it.) Facebook's Hack is another example of this approach -- an overhaul of PHP. Although I guess it led to 2 languages and not one -- it's not replacing PHP! Like I said, it's hard :)
deleted 8y ago
[deleted]
eecc 8y ago
Vendorization. For every language out there there is a consulting firm or corporation selling tooling and support. This allows business strategies aimed at market capture and monopolization. To business, you’re not a general purpose plumber, you’re just a specially trained installer of a particular brand of water boiler.
- frou_dh 8y ago
  Are you saying that the original creators of languages are motivated to a large extent by the thought of distant future consulting money? Because that does not ring true at all.
- GuB-42 8y ago
  This is a consequence, not a cause. With a few exceptions (ex: Java), successful languages are designed with a use case. C is for writing UNIX, Rust for Firefox, PHP for generating personal home pages and JS was Netscape's solution for dynamic webpages, Go is well suited to what Google is doing. Authors of these languages want them to be successful because the more popular a language is, the more people will help them on their project. They may even give out free tooling and support for that reason. Consulting firms usually come later, when the language is already popular.
- tabtab 8y ago
  I suspect the Razor engine/language is a case of vendorization. (Razor is a markup templating language which is part of MS Visual Studio.) I suspect MS promoted it because it complicates IDE design (parsing) so as to keep other IDE's out of their turf. The prior templating syntax ("<%...%>") was simple to learn, simple to use, easy to read, easy to debug, and easy to parse (per IDE vendor). Razor didn't add much in practical features to justify the huge leap in complexity it caused. It may save 2% of key-strokes but complicated syntax and parsing by a factor of roughly 30. It would have to save at least 25% of keystrokes to justify 30x complexity in my opinion. Let alone debugging and parsing error headaches. Bad deal.
burntsushi 8y ago
I also disagree with the premise of this article, and I do not think the argument presented here is compelling at all. The motivations that spur people to do work are complex and diverse, and this article seems to pretend otherwise. For example: > but it's common for new languages to trigger reimplementation of, e.g., container data structures, HTTP clients, and random number generators. If the new language did not exist, that effort could have been spent on improving existing libraries or some other useful endeavour. Yes, it could have, but would it? Speaking as someone who has done some of this reimplementation work in a newer language, I can unequivocally say that it would not in my case. I'm just one data point, but I don't think I'm unusual. There is an aspect of greenfield development that is appealing to me. It is a somewhat unique opportunity to execute a vision with a lot of freedom. There is also the benefit of new forms of expression that new tools give you. I'm painting with broad strokes here, but I'm generally a believer in the idea that tools themselves can both limit and empower the expression of ideas. If it weren't for the new language, I surely would have done something else with my time. I don't know what it would be, but I know it would not be trying to expend the social capital required to make small incremental improvements to existing software using tools that I find too limited. As another commenter mentioned, I might have just watched more Netflix.
- pron 8y ago
  There are a couple of problems with your perspective: 1. Even if re-implementation of libraries in new languages were free due to an excitement factor, real software is not built this way. New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel. 2. It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work.
  - pythonaut_16 8y ago
    And the flaw with your perspective is that if we all followed this advice, we might have a few highly polished bullet proof C libraries to use, but we'd still be stuck writing C and following all of its paradigms.
alexpw 8y ago
"If the new language did not exist, that effort could have been spent on improving existing libraries or some other useful endeavour." This ignores the value and usefulness of learning for the sake of learning, and to allow other people to hone their skills.
- yxhuvud 8y ago
  Not to mention legacy inertia - things get harder to change in existing libraries when they have loads of users. Even if the change make sense in a vacuum.
siscia 8y ago
I don't agree. With times languages become obsolete, so from one side they try to "upgrade" their syntax, semantic and internal, from the other side they keep backward compatibility. Keep adding features on top of something that was not designed for those features will simply create a huge mess to work with. And arguably the oldest languages are the one where you need to follow "best practise", where there are thousands of way to achieve the same goal, etc... It is not bad, but we really need newer languages to move forward the field. Most of them will fail, some of them will fail while having used a great amount of resources, but some will eventually succeed and moving the field forward.
- zzzcpan 8y ago
  I don't think languages progress evolutionary like species. None will probably succeed, but we will have a lot more variety of languages for anything we can think of.
fouc 8y ago
How about the reverse problem? The cost of mindshare. Everyone grouping up on a major programming language or framework. It's a bit irritating when people often choose their language or framework because of the size of the community rather than actually evaluating alternative languages or frameworks first. Take React for example. There are some great alternatives to it, especially inspired by react but simpler & cleaner DSLs. But people just keep with React anyways because everyone is already there.
- jondubois 8y ago
  That's true but the learning curve difference between React and VueJS isn't anywhere near as significant as the learning curve difference between JavaScript and Elixir for example. Programming languages have a lot of nuances and patterns of use which can take years to fully absorb. Angular, VueJS and React share many similar patterns and best-practices.
- ttty 8y ago
  I used react for 4 years. I don't see the point in using something that is 10% different. Then you need to hire engineers, most of them know more react than an obscure library. React has lower chance to be discontinued. React works well. I rather focus on building new products than changing tech. Of course during the jQuery time I hated it all the time and I was looking for alternatives, but react is exactly how I think about UI: components (html, CSS, js all in one file) and not templates. I'm pretty efficient at react and I can't justify using something else.
  - madmaniak 8y ago
    Using React you just have no idea what efficiency is.
    - ttty 8y ago
      Can you further explain what you mean?
      - 8y ago
jondubois 8y ago
It's annoying how new languages sometimes become popular in spite of not adding any value to the development process. I think part of the problem is that new languages open up a new market which incentivises developers to create open source libraries that will become the foundation of the new ecosystem; it's a sure way to become a 'rockstar developer'; if the language succeeds, you succeed with it; the odds of success are much higher than having to compete with other libraries and frameworks within a well established ecosystem.
flohofwoe 8y ago
I really believe that it is better to have many small 'throw-away' programming languages that can interoperate with each other, than having a few huge languages that are isolated in their own ecosystem. A language shouldn't be complex nor should it take more than a few days to learn, and it should be easy to abandon when another language is better suited for a problem. What actually makes many languages useful is their standard libraries and library ecosystem, and the two always get mixed up. When people talk about how great language X is, in most cases they mean how feature-rich or easy to use the standard library of that language is. With few exceptions, those libraries shouldn't be tied to a specific language. Let me access your library written in language X from my language Y. Let me easily create projects that are made of different languages, for each part of the project the language that fits this part best.
- alexhutcheson 8y ago
  Function calls between languages are typically expensive and difficult to implement. Each language generally has its own expectations about the layout of data in memory, which means that for language A to call a function written in language B, it needs to set up a block of memory in the layout expected by language B. This normally involves a lot of copying, which adds overhead to the function call. There also needs to be some code that does this copying, and this code needs to be implemented for every pair of languages. Alternatively, all the languages can agree on a common interface level (C, JVM, .NET, etc.), but in this case most languages won't get a function interface that feels native to the language.
  - flohofwoe 8y ago
    Agreed, but I think those problems should be solved, the same way LLVM solved the N:M frontend:backend problem :) It's very likely that interacting with such a generic library interface doesn't feel native to the language, but I think it doesn't have to be a monstrosity like DOM or CORBA.
    - PeterisP 8y ago
      IMHO these problems are fundamentally unsolvable, often because the data models are incompatible (the fundamental behavior and benefits of language A relies on their data structures having trait X; and the fundamental behavior and benefits of language B relies on their data structures not having trait X), so a library written in one language can't just accept nontrivial data from the other, it can't be allowed to operate directly on the other language's data in memory without at least copying it and often requires a performance-killing transformation of the whole data. You see that in all kinds of interfaces across language boundaries. Memory layouts, thread safety, (im)mutability, structure ownership or reference counting, handles to external/OS resources, etc, etc. Every language is an abstraction that relies on certain assumptions, and those particular assumptions are what makes the language good for it's niche. Different languages rely on different, often incompatible assumptions. It's possible to build a wrappers that ensure that the assumptions of the other language are met and interaction with native data structures is possible and convenient (e.g. like Numpy does for Python), but that wrapper needs to be different for different languages and assumptions, it can't be the same library, it needs to behave differently in other languages to meet their expectations.
digitalzombie 8y ago
I disagree. They can do whatever they want because it's passion driven. If people finds the new language wonderful they'll choose to spend their time there and to create their own community around it. It's their free time and they have every right to choose how to spend it. Also with RPC, the apache project arrow, etc... there'll always be people out there will bridge community. Also is this really a problem? Programming language fragmentation? I've seen front end javascript frameworks fragmentation everywhere and they're fine with it. And the solution on the horizon are standardization such as web component (via w3c). These frameworks converge toward web component and now it's getting standardize. I'd argued that because of the fragmentation we actually know what we want to standardize and it also pushed for it. If there weren't any fragmentation I'm not entirely sure if web component would ever be standardize.
- jcelerier 8y ago
  > I've seen front end javascript frameworks fragmentation everywhere and they're fine with it just because they're fine with it does not mean that everyone is fine with it
  - codr4 8y ago
    And that's fine too, it's perfectly ok to disagree with the choices others make. But that's as far as it goes, you wouldn't like someone else telling you how to spend your time without getting anything in return either. I'd bet most developers would be fine with supporting and incrementally improving if they made enough money from that to do what they really want to do on the side. I would take a minute or two to reflect on all the time and energy that hundreds of thousands of individuals give away for free. It's easy to forget how far we've come by standing on their shoulders.
- gameswithgo 8y ago
  you are not disagreeing with the article.
- deleted 8y ago
  [deleted]
PedroBatista 8y ago
Thank god nobody listened to his advice, otherwise we all be doing Basic and Java ( such rebel’s)
wwwigham 8y ago
Fragmentation yields a distribution of effort which yields theoretically inferior products, compared to the theoretical output of all the individual parts working towards the same goal. (Debatable, but we'll assume it.) Centralization yields a lack of competition yields a lack of a drive to improve inferior products yields stagnation and disenfranchisement. (Also debatable, but probably safe to assume here as well.) Startup costs in this area (basic compiler stuff, usually handled by llvm and other varied metacompiler frameworks nowadays) become vanishingly small with time, while the long tail becomes ever larger (libraries, tooling, ecosystem goodness, all are expected now). Given these two observations, when a language becomes popular despite starting in a fragmented ecosystem, it slowly grows, starts to take advantage of network effects, and gains the benefits of growing centralization as it becomes "the defacto choice" in some area (usually at the cost of other players in the space, be they large or small). Eventually, it stagnates, parts of the community become disenfranchised and go and spawn their own languages and variants (taking ideas from their origin, along with their grievances and ideas from other areas with them), and the process begins anew. However because of the growing long tail, each time this cycle takes place, the time between expansion and explosion takes longer and longer. At least, that's how I've been led to believe systems like these tend to work. It seems like the call to be _mindful_ of what you're doing when you make a language is sensible; if you contribute to the cycle, you're contributing to what will eventually be 2 years of long-tail work for what will be the de-facto norm for developer tooling 40 years from now; but that's a terrible way to look at it. Wouldn't you rather get in closer to the ground floor and be part of the first iterations that set the standards for the iterations which come after? Isn't thinking of it any other way just being defeatist? (Since it amounts to concluding that future generations of developers must be better at this than you, and so are worth more early-cycle-iteration time?) So what if it's indulging the ego some; if that's the cost of improvement, then so be it. The languages we have today wouldn't have been made without their predecessors, and the languages of tomorrow won't exist without the ones we have today. Is it also equally possible that deciding _not_ to make some new language potentially delaying the progress of the programming language field as a whole? Is making that value judgement in the purview of anything other than hindsight? I can't begin to imagine in what ways future languages may improve life as a developer, but if they're anything like the stark contrasts we've seen recently in certain PL areas (ie, systems languages), it's sure to be exciting.
cntlzw 8y ago
So, if we follow the authors logic we should abandon all science because we don't know what they might cause. Innovation is pure by nature. Judgment is done empirical evidence after innovation. You can't change the order.
- ken 8y ago
  No. Languages are also, of course, languages, i.e., notation. If every chemist changed notation every 3 years, there's an off chance that one might develop something better than Hill notation, but it would also make it unbearable to try to read the literature. I don't think it's controversial to suggest that this would be a net loss for the field. Computer science is young enough that we don't have any universal notation yet, but I've learned a couple dozen programming languages and my conclusion is that maybe 1 new language per decade is worth using. We're churning notation way faster than is productive.
- onemoresoop 8y ago
  I disagree with your statement. It would in fact be: scientists create unnecessary new notations and that causes a lot of fragmentation in the science community. They all try to solve one thing which is do science. On the other hand I would agree that some new notation could create new ways to see things that were not possible with the old notation. I think in the long run we're ok with many programming languages. Whatever is useful will survive and what isn't will die out.
toolslive 8y ago
It's a very good question. However, thinking about it, the conclusion should probably be that the cost of continuing to use the same language while there exists another language more suitable to attack your problem is higher.
fauigerzigerk 8y ago
Isn't this a pretty trivial case of exploration vs exploitation? Clearly we need both. The more interesting question for me is why we still don't have a widely adopted approach to sharing libraries across languages. The most promising approach I have seen in recent decades was Microsoft's COM. But it has declined along with Microsoft's clout. Unfortunately, current industry leaders don't seem to be interested in this problem at all.
- zzzcpan 8y ago
  > why we still don't have a widely adopted approach to sharing libraries across languages We have C, lots of languages use C libraries. Pick a random language and it probably has OpenSSL bindings.
  - fauigerzigerk 8y ago
    That's great for sharing libraries written in C. It doesn't generally solve the problem of reusing libraries across languages though.
  - sevensor 8y ago
    Not only that, we have the Java and .NET runtimes, both of which can be used from many languages. I'd say, along with C, that makes three widely adopted approaches to sharing libraries between languages. Furthermore, C libraries don't even have to be written in C. It's far from unheard-of to use a more powerful language like a lisp or Ocaml to write a C library.
- Svenskunganka 8y ago
  Maybe the GraalVM is what you're looking for? https://www.graalvm.org/ https://www.graalvm.org/
- kazinator 8y ago
  > widely adopted approach to sharing libraries across languages For platform libraries we do; it's called FFI. Your language can use openssl or libxml or whatever, so can mine. It is more general than COM; COM is a specific ABI (that we can target with FFI). Language-specific libraries are tied to language semantics, because language-specific objects pass across language-specific calling boundaries. We don't think about using Python dictionaries from Guile Scheme and such.
sklogic 8y ago
Another one failing to comprehend that different languages have different application domains. Sad.
stellalo 8y ago
When exactly have diversity and choice become problems? It’s a blessing that so many new languages are popping up. Those that have a reason to exist, will succeed, and the time spent on them will be an investment; those that will fail will do so for a reason. And that’s ok. Some people will have wasted their time. But not the community as a whole. Failures are part of evolution. Thanks to failures we have Rust and Julia and Nim and who knows how many other new interesting languages.
zzzcpan 8y ago
Programming languages and libraries are not universally expressive or useful and are not of universally good quality. New independent languages and reimplementations are necessary to better express problems and therefore reduce defects and maintenance costs. They are also necessary for anything of quality to emerge, for all the right people to make all the right choices. Incidentally this is one of the reasons that prevents large projects from achieving good quality and large organizations from producing quality software.
- MrEfficiency 8y ago
  Im not sure where javascript falls along this. A painful language that I can use everywhere exists.
amirouche 8y ago
Thankfully most of the comments here on HN argue: a) Your advice on how I spend my free time is not welcome, b) Same ideas that apply to science apply to FLOSS code. You can do and share you research. And completely disagree with the idea that if I publish some code I do a commitment to maintain that code. That's written in most (all?) FLOSS license: this software comes with no warranty.
michaelfeathers 8y ago
In ecology, what the author cites as a problem would be seen as a sign of a healthy robust ecosystem.
indymike 8y ago
Most new languages come with an improvement, usually of the non-trivial sort and grow communities and survive on their own merit. Usually, the benefit of a new language exceeds the cost... or it simply gets no traction.
zimbatm 8y ago
How much of the new languages are supported by a new fresh wave of engineers that want to learn things? To become a good engineer I had to go through implementing my own container types, play with my own little databases and network libraries, implement a build system, ... I published a few of those experiments in Ruby, the new/hip language at the time. Picking a fresh language seems the obvious playground to do this, it's where you can leave your mark. My impression was that nodejs developers' age distribution was quite young as well when it started. My point is that it's necessary to introduce new languages once it a while to get good engineers :)
- bunderbunder 8y ago
  Tangential anecdote: At one company I worked for, the guy in charge of ops had put a whole lot of work into standardizing the production environment and tooling, and getting it all tuned just so. A couple years later, he lamented that new hires were achieving deep competence more slowly, and weren't reaching the same level of competence that previous team members used to. It wasn't a big deal most the time, but they made a lot of mistakes when rolling out new components, and tended to get caught flat-footed whenever something went sideways. He suspected that the problem was that, in its current state, the new system worked so smoothly that they could get away with understanding things at a very superficial level, and that hindered their learning.
- BeetleB 8y ago
  That was my first thought when I read the post: He's discounting the fact that by doing this, many programmers are gaining in expertise. It's not a given that those programmers would grow their skill set as much otherwise. Advancing the state of the art in a well established language requires a lot more skills than implementing simple libraries for a new language.
  - mike_hearn 8y ago
    Yeah, but they could write potentially useful apps or features, instead of badly re-writing libraries that already exist.
lazyjones 8y ago
He‘s right about the cost, but it‘s surely smaller than the cost of bloated protocols, specifications, reference implementations. Nobody except large corporations is able to implement a competitive web browser, for example, regardless of the language. New languages would be much less of an issue with clean API designs and specifications.
deleted 8y ago
[deleted]
kazinator 8y ago
Developing TXR is one of the best things I've done. From the perspective of a user, I'm grateful for its existence.
- wdkrnls 8y ago
  I'm grateful too.
honkycat 8y ago
The opinions expressed in this blog post are one of those uninteresting, infinitely parroted idioms that people like to assert to seem smart and pragmatic, but are completely idiotic upon further reflection. It's similar to "If you rent you are throwing your money away" completely ignoring the realities of how DUMB that statement is. Paying for a house which you must mortgage, pay taxes on, and maintain, but cannot rent out or sell is much more expensive than renting someone else's house, and historically the stock market gives greater yields for many investors. First of all, we have not yet written the perfect programming language, and every single language written has pros and cons for various different tasks. Choosing the right language can lead to writing abstractions that make you ABSURDLY more productive than alternative choices. Write RabbitMQ in Javascript instead of Erlang and tell me we only need one language to rule them all. Off the top of my head, I can think of 5 languages that have MASSIVELY improved the software ecosystem over the last 10 years: Golang, Rust, Elixir, Typescript, Elm. > However, I hope people consider carefully the social costs of creating a new programming language especially if it becomes popular, and understand that in some cases creating a popular new language could actually be irresponsible. Unsurprising a self-proclaimed christian programmer takes the moral high ground, and asserts writing a new programming language out of passion, love, or an actual need for a new language to allow for cleaner abstractions for your particular use case can POSSIBLY be unethical. I have a different suggestion: Consider the social costs of parroting idiotic drivel you heard one time and trying to pass it off as actually interesting insight.
hpstrkng 8y ago
It's like asking authors to not write books unless they know they have a New York Times bestseller on their hands...
preordained 8y ago
I think this is one of the best things about Clojure and similar "hosted" languages...leveraging preexisting huge ecosystems. You really do (as much as one could reasonably expect) get to have your cake and eat it too.
casper345 8y ago
Points to the author, just because one can create doesn't mean one should. Should there not be more intentional thought in creating and creations ramifications? "Now I am become Death, the destroyer of worlds."
- zsck 8y ago
  You can’t seriously be comparing rebuilding arbitrary software to building an atomic weapon.
glangdale 8y ago
My "side" take on this is that as long as reimplementation is considered an urgent activity with each new language, we may be privileging languages that are actually better for rebuilding things that are already well understood over languages that are better for "feeling our way through problems". I'm not 100% sure of this point, but it feels at least possible that features that make a language good for reimplementing a problem with the structure known in advance might be awkward for exploratory work. I've built a bunch of stuff recently where I had NFI what I was doing as I went along and had to tear up vast tracts of the data structures as I went; it was good to have the fairly loosey-goosey typing of a mix of C/C++ and (effectively) asm to do it (while still being able to see performance levels of aggressive optimization, which was also part of the goal).

137 points by jp_sc 8y ago

That reflects rather well how I feel about the recent DSL craze, specially in the ruby community. It's also how I feel about libraries and frameworks. Programming languages probably suffer the least from this phenomenon due to how difficult it is to create a complete language, with compilers and all. In contrast, languages that just extend others often do benefit from existing communities. Take moonscript¹ for example, it's just a new syntax for Lua, so you can use all the existing libraries at no cost. Or take Terra², which can make use of all the C libraries out there and the Lua libraries at the same time. ¹ http://moonscript.org/ http://moonscript.org/ ² http://terralang.org/ http://terralang.org/

flukus 8y ago
Is the DSL craze really new? Look at a lot of the early history of unix and you'll see sh, sed, awk, tex and probably a heap of others that didn't survive. Not to mentions lex/yacc, two DSL's to make DSLs. The idea of specialized languages for specialized tasks has been around for decades.
smadge 8y ago
An example that comes up often for me are EDSLs for database queries. I do see the value of them, but on the other hand it causes you to move away from a “lingua Franca” of database querying to a programming environment specific one. What probably makes sense is 1) evolving SQL or SQL tooling to support more features that people think they need (an example that comes to mind is type safe queries) and 2) better support for binding these queries to different languages.
- lmm 8y ago
  External DSLs (even with "cross-language bindings") are much worse to work with than than internal ones and SQL is no exception. IMO what's really needed is a willingness for databases to step away from SQL and expose an API that's more friendly to modern programming languages (or better yet, make the database embeddable as a library rather than a framework you have to build your application into). The eDSLs help a little, but as long as they're obliged to compile into SQL strings there's a limit to how effective they can be.
vidarh 8y ago
Most of "the DSL craze" in Ruby is just defining methods. It's rarely new languages, but simply a matter of taking full advantage of the language to let you talk about your domain in a more natural way. It's not so much a "dsl craze" in Ruby as writing idiomatic Ruby well, the same was as e.g. Smalltalk or lisps also tend to make you create your own vocabulary that is close to indistinguishable (more so than in Ruby, in fact, where the block syntax and control structure syntax differ) from the language itself.
lassejansen 8y ago
Not sure if "recent" is the correct attribute for DSLs in Ruby. https://www.infoq.com/news/2007/06/dsl-or-not https://www.infoq.com/news/2007/06/dsl-or-not

michaelmrose 8y ago

"However, I hope people consider carefully the social costs of creating a new programming language especially if it becomes popular, and understand that in some cases creating a popular new language could actually be irresponsible." Passion isn't fungible and there is zero gauruntee that the person who creates an interesting new language wouldn't have chosen to binge watch Netflix instead. Furthermore without a crystal ball it's difficult to separate ahead of time which efforts will move us forward in some small or large ways. The entire premise of this post is flawed.

jacquesm 8y ago
> it's difficult to separate ahead of time which efforts will move us forward in some small or large ways. That's the point: most efforts do not move us forward but instead move us backwards because fewer people are working on the things that matter. It's a solid argument and it applies to far more of the open source community than just to programming languages, in fact programming languages are the smaller part of the issue, but the argument still applies. > The entire premise of this post is flawed. I don't think so. I would not dismiss an article without at least trying to make a genuine effort to understand the point an article makes. The fact is that it takes effort to launch a new programming language beyond just writing some code and the long term commitment should be there if you are going to let other people run their production systems on what you throw into the world. This doesn't mean you don't get to scratch your itch, it means that once your programming language gains adoption beyond some people playing around with it you can't just walk off and say that it isn't your problem. Oh, and it is 'guarantee'.
- a_imho 8y ago
  The fact is that it takes effort to launch a new programming language beyond just writing some code and the long term commitment should be there if you are going to let other people run their production systems on what you throw into the world. From my admittedly limited knowledge JavaScript seems to contradict this claim. I don't agree with the article either, one could make the same argument for every piece of new code written.

alexmorley 8y ago

I understand the concern but it assumes that open-source contributions are a zero-sum game, which I don't think they are. New languages often encourage users to become contributors where they might have remained users in the old language [citation needed]

ChrisCinelli 8y ago

It is not just creating a new language. In a company environment you see a huge cost when people want to start using a new language or/and framework. It comes down to values and what push people forward. I am not saying one language should be enough. I am saying that for example that even for a company that already has a library of more than a few components built in React a a medium codebase using it, starting using Vue will add a quite large cost. Even if a framework is 20% better in some metrics, having to rewrite some reusable components (assuming they were written well) in another framework is going to sunk a bunch of time. There is a learning curve that people need to go through. Hardly the first time you use a framework, you end up writing the most awesome code in that framework. There are bugs, there may not be a lot of documentation or open source libraries. From an engineering prospective is going to be fun and you are going to learn something new but from a business prospective it is rarely a good decision. Same thing is probably true when teams in company with a large Java codebase want to switch to Scala. On the other end when you have a 10x better framework or technology you are better paying attention and make sure you plan a transition sooner than later. For example, I think GraphQl is a order of magnitude better than REST when you consider a whole end-to-end system. Engineers, especially the smart ones, get bored quickly. Inventing a new largely adopted language or framework is an irresistible calling from inside. Exploring new territories is also another source of growth. It is all about values and the way we go about satisfying them.

TheAceOfHearts 8y ago

I think this could be mitigated in some cases with better tooling. Many programming languages just make it needlessly difficult to interact with anything that's not part of their ecosystem. Check out parcel [0], a web application bundler. It has built-in support for lots of different assets [1], with two notable inclusions being ReasonML and Rust! In this blog post [2] they highlight how easy it is to import Rust code from JavaScript. Another neat example is Objective-C bridging on macOS [3]. The code usually doesn't end up looking very pretty, and it can be brittle at times, but with JavaScriptObjC you can interact directly with all the native APIs using JavaScript. Here's a blog post [4] showing how to write a native app on macOS using JavaScriptObjC. [0] https://github.com/parcel-bundler/parcel https://github.com/parcel-bundler/parcel [1] https://parceljs.org/assets.html https://parceljs.org/assets.html [2] https://medium.com/@devongovett/parcel-v1-5-0-released-source-maps-webassembly-rust-and-more-3a6385e43b95 https://medium.com/@devongovett/parcel-v1-5-0-released-sourc... [3] https://developer.apple.com/library/archive/documentation/LanguagesUtilities/Conceptual/MacAutomationScriptingGuide/HowMacScriptingWorks.html#//apple_ref/doc/uid/TP40016239-CH73-SW4 https://developer.apple.com/library/archive/documentation/La... [4] https://tylergaw.com/articles/building-osx-apps-with-js/ https://tylergaw.com/articles/building-osx-apps-with-js/

reacweb 8y ago

IMHO, the author is completely wrong and the sentence "C is the desert island language" is closer to reality (see http://www-cs-students.stanford.edu/~blynn/c/intro.html http://www-cs-students.stanford.edu/~blynn/c/intro.html). The fact that such a poor language remains the single sane choice to build the kernel of Linux is a proof of the lack of languages. C++ was already a mess in 98. Python started as simple but has added more and more complex syntaxes. Java is in the hands of Satan, Ada is too verbose, nobody takes the effort to learn it, Haskell and SCALA are niche languages not so easy to learn... IMHO, there is a need for a polyvalent language that would enable development of Linux kernel (low level and performance), that would be easy to learn and that would enable quick development (dynamic typing, syntactic sugar, ...).

jacquesm 8y ago
> there is a need for a polyvalent language that would enable development of Linux kernel (low level and performance), that would be easy to learn and that would enable quick development That's a pretty tall order for a single language to support all those desires. But Rust seems to at least aim for that space (it doesn't check all your boxes though).
pjc50 8y ago
Dynamic typing and garbage collection go directly against the "low level and performant" goals. You can avoid GC with clever lifetime rules (Rust) but that loses "easy to learn".
nineteen999 8y ago
I'm not sure how old that article is, but parts of gcc suite have been rewritten in a subset of C++ for several years now (https://lwn.net/Articles/542457/ https://lwn.net/Articles/542457/), starting from over 10 years ago.
- reacweb 8y ago
  Yes, I know that. C++ is also very much used for game development. C++ allows to build very impressive abstractions that are very handy. The main drawback is that this flexibility has a huge impact on the language. When you switch between two projects writen in C++, it is almost as if they were using a different language. I have understood that this language was a mess in 98 when I have bought the standard and read it completely. In professional projects, I always try to avoid it in favor of java despite all the years I have spent learning and using C++.

chubot 8y ago

It's hard to do, but I would like to see more overhauls of existing languages. I wrote a relevant comment about the approach of Checked C vs. Rust here: https://news.ycombinator.com/item?id=17944310 https://news.ycombinator.com/item?id=17944310 (tl;dr Software gets rewritten much less frequently than you think. Rust is going to add to the existing ecosystem in C++ and C, not replace it.) Facebook's Hack is another example of this approach -- an overhaul of PHP. Although I guess it led to 2 languages and not one -- it's not replacing PHP! Like I said, it's hard :)

deleted 8y ago

[deleted]

eecc 8y ago

Vendorization. For every language out there there is a consulting firm or corporation selling tooling and support. This allows business strategies aimed at market capture and monopolization. To business, you’re not a general purpose plumber, you’re just a specially trained installer of a particular brand of water boiler.

frou_dh 8y ago
Are you saying that the original creators of languages are motivated to a large extent by the thought of distant future consulting money? Because that does not ring true at all.
GuB-42 8y ago
This is a consequence, not a cause. With a few exceptions (ex: Java), successful languages are designed with a use case. C is for writing UNIX, Rust for Firefox, PHP for generating personal home pages and JS was Netscape's solution for dynamic webpages, Go is well suited to what Google is doing. Authors of these languages want them to be successful because the more popular a language is, the more people will help them on their project. They may even give out free tooling and support for that reason. Consulting firms usually come later, when the language is already popular.
tabtab 8y ago
I suspect the Razor engine/language is a case of vendorization. (Razor is a markup templating language which is part of MS Visual Studio.) I suspect MS promoted it because it complicates IDE design (parsing) so as to keep other IDE's out of their turf. The prior templating syntax ("<%...%>") was simple to learn, simple to use, easy to read, easy to debug, and easy to parse (per IDE vendor). Razor didn't add much in practical features to justify the huge leap in complexity it caused. It may save 2% of key-strokes but complicated syntax and parsing by a factor of roughly 30. It would have to save at least 25% of keystrokes to justify 30x complexity in my opinion. Let alone debugging and parsing error headaches. Bad deal.

burntsushi 8y ago

I also disagree with the premise of this article, and I do not think the argument presented here is compelling at all. The motivations that spur people to do work are complex and diverse, and this article seems to pretend otherwise. For example: > but it's common for new languages to trigger reimplementation of, e.g., container data structures, HTTP clients, and random number generators. If the new language did not exist, that effort could have been spent on improving existing libraries or some other useful endeavour. Yes, it could have, but would it? Speaking as someone who has done some of this reimplementation work in a newer language, I can unequivocally say that it would not in my case. I'm just one data point, but I don't think I'm unusual. There is an aspect of greenfield development that is appealing to me. It is a somewhat unique opportunity to execute a vision with a lot of freedom. There is also the benefit of new forms of expression that new tools give you. I'm painting with broad strokes here, but I'm generally a believer in the idea that tools themselves can both limit and empower the expression of ideas. If it weren't for the new language, I surely would have done something else with my time. I don't know what it would be, but I know it would not be trying to expend the social capital required to make small incremental improvements to existing software using tools that I find too limited. As another commenter mentioned, I might have just watched more Netflix.

pron 8y ago
There are a couple of problems with your perspective: 1. Even if re-implementation of libraries in new languages were free due to an excitement factor, real software is not built this way. New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel. 2. It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work.
- pythonaut_16 8y ago
  And the flaw with your perspective is that if we all followed this advice, we might have a few highly polished bullet proof C libraries to use, but we'd still be stuck writing C and following all of its paradigms.

alexpw 8y ago

"If the new language did not exist, that effort could have been spent on improving existing libraries or some other useful endeavour." This ignores the value and usefulness of learning for the sake of learning, and to allow other people to hone their skills.

yxhuvud 8y ago
Not to mention legacy inertia - things get harder to change in existing libraries when they have loads of users. Even if the change make sense in a vacuum.

siscia 8y ago

I don't agree. With times languages become obsolete, so from one side they try to "upgrade" their syntax, semantic and internal, from the other side they keep backward compatibility. Keep adding features on top of something that was not designed for those features will simply create a huge mess to work with. And arguably the oldest languages are the one where you need to follow "best practise", where there are thousands of way to achieve the same goal, etc... It is not bad, but we really need newer languages to move forward the field. Most of them will fail, some of them will fail while having used a great amount of resources, but some will eventually succeed and moving the field forward.

zzzcpan 8y ago
I don't think languages progress evolutionary like species. None will probably succeed, but we will have a lot more variety of languages for anything we can think of.

fouc 8y ago

How about the reverse problem? The cost of mindshare. Everyone grouping up on a major programming language or framework. It's a bit irritating when people often choose their language or framework because of the size of the community rather than actually evaluating alternative languages or frameworks first. Take React for example. There are some great alternatives to it, especially inspired by react but simpler & cleaner DSLs. But people just keep with React anyways because everyone is already there.

jondubois 8y ago
That's true but the learning curve difference between React and VueJS isn't anywhere near as significant as the learning curve difference between JavaScript and Elixir for example. Programming languages have a lot of nuances and patterns of use which can take years to fully absorb. Angular, VueJS and React share many similar patterns and best-practices.
ttty 8y ago
I used react for 4 years. I don't see the point in using something that is 10% different. Then you need to hire engineers, most of them know more react than an obscure library. React has lower chance to be discontinued. React works well. I rather focus on building new products than changing tech. Of course during the jQuery time I hated it all the time and I was looking for alternatives, but react is exactly how I think about UI: components (html, CSS, js all in one file) and not templates. I'm pretty efficient at react and I can't justify using something else.
- madmaniak 8y ago
  Using React you just have no idea what efficiency is.
  - ttty 8y ago
    Can you further explain what you mean?
    - 8y ago

jondubois 8y ago

It's annoying how new languages sometimes become popular in spite of not adding any value to the development process. I think part of the problem is that new languages open up a new market which incentivises developers to create open source libraries that will become the foundation of the new ecosystem; it's a sure way to become a 'rockstar developer'; if the language succeeds, you succeed with it; the odds of success are much higher than having to compete with other libraries and frameworks within a well established ecosystem.

flohofwoe 8y ago

I really believe that it is better to have many small 'throw-away' programming languages that can interoperate with each other, than having a few huge languages that are isolated in their own ecosystem. A language shouldn't be complex nor should it take more than a few days to learn, and it should be easy to abandon when another language is better suited for a problem. What actually makes many languages useful is their standard libraries and library ecosystem, and the two always get mixed up. When people talk about how great language X is, in most cases they mean how feature-rich or easy to use the standard library of that language is. With few exceptions, those libraries shouldn't be tied to a specific language. Let me access your library written in language X from my language Y. Let me easily create projects that are made of different languages, for each part of the project the language that fits this part best.

alexhutcheson 8y ago
Function calls between languages are typically expensive and difficult to implement. Each language generally has its own expectations about the layout of data in memory, which means that for language A to call a function written in language B, it needs to set up a block of memory in the layout expected by language B. This normally involves a lot of copying, which adds overhead to the function call. There also needs to be some code that does this copying, and this code needs to be implemented for every pair of languages. Alternatively, all the languages can agree on a common interface level (C, JVM, .NET, etc.), but in this case most languages won't get a function interface that feels native to the language.
- flohofwoe 8y ago
  Agreed, but I think those problems should be solved, the same way LLVM solved the N:M frontend:backend problem :) It's very likely that interacting with such a generic library interface doesn't feel native to the language, but I think it doesn't have to be a monstrosity like DOM or CORBA.
  - PeterisP 8y ago
    IMHO these problems are fundamentally unsolvable, often because the data models are incompatible (the fundamental behavior and benefits of language A relies on their data structures having trait X; and the fundamental behavior and benefits of language B relies on their data structures not having trait X), so a library written in one language can't just accept nontrivial data from the other, it can't be allowed to operate directly on the other language's data in memory without at least copying it and often requires a performance-killing transformation of the whole data. You see that in all kinds of interfaces across language boundaries. Memory layouts, thread safety, (im)mutability, structure ownership or reference counting, handles to external/OS resources, etc, etc. Every language is an abstraction that relies on certain assumptions, and those particular assumptions are what makes the language good for it's niche. Different languages rely on different, often incompatible assumptions. It's possible to build a wrappers that ensure that the assumptions of the other language are met and interaction with native data structures is possible and convenient (e.g. like Numpy does for Python), but that wrapper needs to be different for different languages and assumptions, it can't be the same library, it needs to behave differently in other languages to meet their expectations.

digitalzombie 8y ago

I disagree. They can do whatever they want because it's passion driven. If people finds the new language wonderful they'll choose to spend their time there and to create their own community around it. It's their free time and they have every right to choose how to spend it. Also with RPC, the apache project arrow, etc... there'll always be people out there will bridge community. Also is this really a problem? Programming language fragmentation? I've seen front end javascript frameworks fragmentation everywhere and they're fine with it. And the solution on the horizon are standardization such as web component (via w3c). These frameworks converge toward web component and now it's getting standardize. I'd argued that because of the fragmentation we actually know what we want to standardize and it also pushed for it. If there weren't any fragmentation I'm not entirely sure if web component would ever be standardize.

jcelerier 8y ago
> I've seen front end javascript frameworks fragmentation everywhere and they're fine with it just because they're fine with it does not mean that everyone is fine with it
- codr4 8y ago
  And that's fine too, it's perfectly ok to disagree with the choices others make. But that's as far as it goes, you wouldn't like someone else telling you how to spend your time without getting anything in return either. I'd bet most developers would be fine with supporting and incrementally improving if they made enough money from that to do what they really want to do on the side. I would take a minute or two to reflect on all the time and energy that hundreds of thousands of individuals give away for free. It's easy to forget how far we've come by standing on their shoulders.
gameswithgo 8y ago
you are not disagreeing with the article.
deleted 8y ago
[deleted]

PedroBatista 8y ago

Thank god nobody listened to his advice, otherwise we all be doing Basic and Java ( such rebel’s)

wwwigham 8y ago

Fragmentation yields a distribution of effort which yields theoretically inferior products, compared to the theoretical output of all the individual parts working towards the same goal. (Debatable, but we'll assume it.) Centralization yields a lack of competition yields a lack of a drive to improve inferior products yields stagnation and disenfranchisement. (Also debatable, but probably safe to assume here as well.) Startup costs in this area (basic compiler stuff, usually handled by llvm and other varied metacompiler frameworks nowadays) become vanishingly small with time, while the long tail becomes ever larger (libraries, tooling, ecosystem goodness, all are expected now). Given these two observations, when a language becomes popular despite starting in a fragmented ecosystem, it slowly grows, starts to take advantage of network effects, and gains the benefits of growing centralization as it becomes "the defacto choice" in some area (usually at the cost of other players in the space, be they large or small). Eventually, it stagnates, parts of the community become disenfranchised and go and spawn their own languages and variants (taking ideas from their origin, along with their grievances and ideas from other areas with them), and the process begins anew. However because of the growing long tail, each time this cycle takes place, the time between expansion and explosion takes longer and longer. At least, that's how I've been led to believe systems like these tend to work. It seems like the call to be _mindful_ of what you're doing when you make a language is sensible; if you contribute to the cycle, you're contributing to what will eventually be 2 years of long-tail work for what will be the de-facto norm for developer tooling 40 years from now; but that's a terrible way to look at it. Wouldn't you rather get in closer to the ground floor and be part of the first iterations that set the standards for the iterations which come after? Isn't thinking of it any other way just being defeatist? (Since it amounts to concluding that future generations of developers must be better at this than you, and so are worth more early-cycle-iteration time?) So what if it's indulging the ego some; if that's the cost of improvement, then so be it. The languages we have today wouldn't have been made without their predecessors, and the languages of tomorrow won't exist without the ones we have today. Is it also equally possible that deciding _not_ to make some new language potentially delaying the progress of the programming language field as a whole? Is making that value judgement in the purview of anything other than hindsight? I can't begin to imagine in what ways future languages may improve life as a developer, but if they're anything like the stark contrasts we've seen recently in certain PL areas (ie, systems languages), it's sure to be exciting.

cntlzw 8y ago

So, if we follow the authors logic we should abandon all science because we don't know what they might cause. Innovation is pure by nature. Judgment is done empirical evidence after innovation. You can't change the order.

ken 8y ago
No. Languages are also, of course, languages, i.e., notation. If every chemist changed notation every 3 years, there's an off chance that one might develop something better than Hill notation, but it would also make it unbearable to try to read the literature. I don't think it's controversial to suggest that this would be a net loss for the field. Computer science is young enough that we don't have any universal notation yet, but I've learned a couple dozen programming languages and my conclusion is that maybe 1 new language per decade is worth using. We're churning notation way faster than is productive.
onemoresoop 8y ago
I disagree with your statement. It would in fact be: scientists create unnecessary new notations and that causes a lot of fragmentation in the science community. They all try to solve one thing which is do science. On the other hand I would agree that some new notation could create new ways to see things that were not possible with the old notation. I think in the long run we're ok with many programming languages. Whatever is useful will survive and what isn't will die out.

toolslive 8y ago

It's a very good question. However, thinking about it, the conclusion should probably be that the cost of continuing to use the same language while there exists another language more suitable to attack your problem is higher.

fauigerzigerk 8y ago

Isn't this a pretty trivial case of exploration vs exploitation? Clearly we need both. The more interesting question for me is why we still don't have a widely adopted approach to sharing libraries across languages. The most promising approach I have seen in recent decades was Microsoft's COM. But it has declined along with Microsoft's clout. Unfortunately, current industry leaders don't seem to be interested in this problem at all.

zzzcpan 8y ago
> why we still don't have a widely adopted approach to sharing libraries across languages We have C, lots of languages use C libraries. Pick a random language and it probably has OpenSSL bindings.
- fauigerzigerk 8y ago
  That's great for sharing libraries written in C. It doesn't generally solve the problem of reusing libraries across languages though.
- sevensor 8y ago
  Not only that, we have the Java and .NET runtimes, both of which can be used from many languages. I'd say, along with C, that makes three widely adopted approaches to sharing libraries between languages. Furthermore, C libraries don't even have to be written in C. It's far from unheard-of to use a more powerful language like a lisp or Ocaml to write a C library.
Svenskunganka 8y ago
Maybe the GraalVM is what you're looking for? https://www.graalvm.org/ https://www.graalvm.org/
kazinator 8y ago
> widely adopted approach to sharing libraries across languages For platform libraries we do; it's called FFI. Your language can use openssl or libxml or whatever, so can mine. It is more general than COM; COM is a specific ABI (that we can target with FFI). Language-specific libraries are tied to language semantics, because language-specific objects pass across language-specific calling boundaries. We don't think about using Python dictionaries from Guile Scheme and such.

sklogic 8y ago

Another one failing to comprehend that different languages have different application domains. Sad.

stellalo 8y ago

When exactly have diversity and choice become problems? It’s a blessing that so many new languages are popping up. Those that have a reason to exist, will succeed, and the time spent on them will be an investment; those that will fail will do so for a reason. And that’s ok. Some people will have wasted their time. But not the community as a whole. Failures are part of evolution. Thanks to failures we have Rust and Julia and Nim and who knows how many other new interesting languages.

zzzcpan 8y ago

Programming languages and libraries are not universally expressive or useful and are not of universally good quality. New independent languages and reimplementations are necessary to better express problems and therefore reduce defects and maintenance costs. They are also necessary for anything of quality to emerge, for all the right people to make all the right choices. Incidentally this is one of the reasons that prevents large projects from achieving good quality and large organizations from producing quality software.

MrEfficiency 8y ago
Im not sure where javascript falls along this. A painful language that I can use everywhere exists.

amirouche 8y ago

Thankfully most of the comments here on HN argue: a) Your advice on how I spend my free time is not welcome, b) Same ideas that apply to science apply to FLOSS code. You can do and share you research. And completely disagree with the idea that if I publish some code I do a commitment to maintain that code. That's written in most (all?) FLOSS license: this software comes with no warranty.

michaelfeathers 8y ago

In ecology, what the author cites as a problem would be seen as a sign of a healthy robust ecosystem.

indymike 8y ago

Most new languages come with an improvement, usually of the non-trivial sort and grow communities and survive on their own merit. Usually, the benefit of a new language exceeds the cost... or it simply gets no traction.

zimbatm 8y ago

How much of the new languages are supported by a new fresh wave of engineers that want to learn things? To become a good engineer I had to go through implementing my own container types, play with my own little databases and network libraries, implement a build system, ... I published a few of those experiments in Ruby, the new/hip language at the time. Picking a fresh language seems the obvious playground to do this, it's where you can leave your mark. My impression was that nodejs developers' age distribution was quite young as well when it started. My point is that it's necessary to introduce new languages once it a while to get good engineers :)

bunderbunder 8y ago
Tangential anecdote: At one company I worked for, the guy in charge of ops had put a whole lot of work into standardizing the production environment and tooling, and getting it all tuned just so. A couple years later, he lamented that new hires were achieving deep competence more slowly, and weren't reaching the same level of competence that previous team members used to. It wasn't a big deal most the time, but they made a lot of mistakes when rolling out new components, and tended to get caught flat-footed whenever something went sideways. He suspected that the problem was that, in its current state, the new system worked so smoothly that they could get away with understanding things at a very superficial level, and that hindered their learning.
BeetleB 8y ago
That was my first thought when I read the post: He's discounting the fact that by doing this, many programmers are gaining in expertise. It's not a given that those programmers would grow their skill set as much otherwise. Advancing the state of the art in a well established language requires a lot more skills than implementing simple libraries for a new language.
- mike_hearn 8y ago
  Yeah, but they could write potentially useful apps or features, instead of badly re-writing libraries that already exist.

lazyjones 8y ago

He‘s right about the cost, but it‘s surely smaller than the cost of bloated protocols, specifications, reference implementations. Nobody except large corporations is able to implement a competitive web browser, for example, regardless of the language. New languages would be much less of an issue with clean API designs and specifications.

deleted 8y ago

[deleted]

kazinator 8y ago

Developing TXR is one of the best things I've done. From the perspective of a user, I'm grateful for its existence.

wdkrnls 8y ago
I'm grateful too.

honkycat 8y ago

The opinions expressed in this blog post are one of those uninteresting, infinitely parroted idioms that people like to assert to seem smart and pragmatic, but are completely idiotic upon further reflection. It's similar to "If you rent you are throwing your money away" completely ignoring the realities of how DUMB that statement is. Paying for a house which you must mortgage, pay taxes on, and maintain, but cannot rent out or sell is much more expensive than renting someone else's house, and historically the stock market gives greater yields for many investors. First of all, we have not yet written the perfect programming language, and every single language written has pros and cons for various different tasks. Choosing the right language can lead to writing abstractions that make you ABSURDLY more productive than alternative choices. Write RabbitMQ in Javascript instead of Erlang and tell me we only need one language to rule them all. Off the top of my head, I can think of 5 languages that have MASSIVELY improved the software ecosystem over the last 10 years: Golang, Rust, Elixir, Typescript, Elm. > However, I hope people consider carefully the social costs of creating a new programming language especially if it becomes popular, and understand that in some cases creating a popular new language could actually be irresponsible. Unsurprising a self-proclaimed christian programmer takes the moral high ground, and asserts writing a new programming language out of passion, love, or an actual need for a new language to allow for cleaner abstractions for your particular use case can POSSIBLY be unethical. I have a different suggestion: Consider the social costs of parroting idiotic drivel you heard one time and trying to pass it off as actually interesting insight.

hpstrkng 8y ago

It's like asking authors to not write books unless they know they have a New York Times bestseller on their hands...

preordained 8y ago

I think this is one of the best things about Clojure and similar "hosted" languages...leveraging preexisting huge ecosystems. You really do (as much as one could reasonably expect) get to have your cake and eat it too.

casper345 8y ago

Points to the author, just because one can create doesn't mean one should. Should there not be more intentional thought in creating and creations ramifications? "Now I am become Death, the destroyer of worlds."

zsck 8y ago
You can’t seriously be comparing rebuilding arbitrary software to building an atomic weapon.

glangdale 8y ago

My "side" take on this is that as long as reimplementation is considered an urgent activity with each new language, we may be privileging languages that are actually better for rebuilding things that are already well understood over languages that are better for "feeling our way through problems". I'm not 100% sure of this point, but it feels at least possible that features that make a language good for reimplementing a problem with the structure known in advance might be awkward for exploratory work. I've built a bunch of stuff recently where I had NFI what I was doing as I went along and had to tear up vast tracts of the data structures as I went; it was good to have the fairly loosey-goosey typing of a mix of C/C++ and (effectively) asm to do it (while still being able to see performance levels of aggressive optimization, which was also part of the goal).