8 ms·

I also disagree with the premise of this article, and I do not think the argument presented here is compelling at all. The motivations that spur people to do wo

I also disagree with the premise of this article, and I do not think the argument presented here is compelling at all. The motivations that spur people to do work are complex and diverse, and this article seems to pretend otherwise. For example: > but it's common for new languages to trigger reimplementation of, e.g., container data structures, HTTP clients, and random number generators. If the new language did not exist, that effort could have been spent on improving existing libraries or some other useful endeavour. Yes, it could have, but would it? Speaking as someone who has done some of this reimplementation work in a newer language, I can unequivocally say that it would not in my case. I'm just one data point, but I don't think I'm unusual. There is an aspect of greenfield development that is appealing to me. It is a somewhat unique opportunity to execute a vision with a lot of freedom. There is also the benefit of new forms of expression that new tools give you. I'm painting with broad strokes here, but I'm generally a believer in the idea that tools themselves can both limit and empower the expression of ideas. If it weren't for the new language, I surely would have done something else with my time. I don't know what it would be, but I know it would not be trying to expend the social capital required to make small incremental improvements to existing software using tools that I find too limited. As another commenter mentioned, I might have just watched more Netflix.

pron 8y ago
There are a couple of problems with your perspective: 1. Even if re-implementation of libraries in new languages were free due to an excitement factor, real software is not built this way. New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel. 2. It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work.
- pythonaut_16 8y ago
  And the flaw with your perspective is that if we all followed this advice, we might have a few highly polished bullet proof C libraries to use, but we'd still be stuck writing C and following all of its paradigms.
  - jfoster 8y ago
    That's the interesting part. We're better off because of some but not all new languages having been invented.
    - runevault 8y ago
      Sure, but you have to have the bad attempts to learn things that don't work or we won't end up with the good ones. We fall into a local maximum instead of exploring farther and potentially going higher. Every dead end is another bit of the map explored when it comes to creating expressive languages. Now, is there an argument we'd be better off doing that exploration in a space designed to make DSLs easier than in building entirely new languages? That I dunno but it is worth considering.
  - deleted 8y ago
    [deleted]
- burntsushi 8y ago
  I don't really appreciate the debate tactic of pretending that I made a stronger claim that I actually did. In particular, I didn't claim or imply that re-implementation of things was "free." It is much much easier to demonstrate that claim as ridiculous as opposed to the more measured perspective that I actually expressed. > real software is not built this way This is a bullshit say-nothing phrase. You're using "real software" to presumably denote some dichotomy of software that doesn't exist. If you asked 10 people to define what "real software" actually meant, you'd get 10 different definitions. They might agree on the extremes; but there's a whole mess of crap in the middle that we could endlessly debate about. > New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel. This isn't clearly true at all. I mean, if you're going to talk about maintenance, then you also need to consider the impact of improved tools on the costs of maintenance. If a tool makes maintenance easier or less costly, then it's pretty hard to say anything definitive about long term costs. New tools can and do impact the usage of old tools, which in turn can have a direct impact on long term maintenance costs. > It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work. Absolutely! But that's not the argument I'm countering. I'm specifically countering the notion that new language development can be "irresponsible" because it "could" divert work away from other projects. What I'm saying is that it is not necessarily true at all. There is a more measured argument to be made here, but the OP did not make it. New ideas should be born and die in accordance with the resources of folks willing to keep them alive. We don't need some grand notion of "responsibility" to control the ebb and flow of ideas.
  - pron 8y ago
    > You're using "real software" to presumably denote some dichotomy of software that doesn't exist. I think that what I meant was clear from context: software that has a long-term, significant impact. > then you also need to consider the impact of improved tools on the costs of maintenance Sure, but however much maintenance costs, it still needs to be done. It cannot be carried long by a novelty factor. > New ideas should be born and die in accordance with the resources of folks willing to keep them alive. We don't need some grand notion of "responsibility" to control the ebb and flow of ideas. I agree. Anyone should work on whatever they find interesting, and that (normally) doesn't make the choice irresponsible. But I still think that empirically new languages, on average, increasingly cost more and contribute less.
empath75 8y ago
There’s a low-hanging fruit element to new languages as well. Reimplementing is a lot easier than improving, since improving a well understood algorithm requires invention and redoing something only requires research.
- burntsushi 8y ago
  Definitely. Also, there can be flaws in the actual implementation of older tools as well. I'm going to avoid diving into specifics because it would be a distraction, but I know more than a few older tools that would benefit from parallelism. They were written in a time where parallelism wasn't as ubiquitously supported as it is today, and thus, the entire implementation would need to be carefully rethought, refactored and possibly rewritten to grow that benefit. We're talking shared global mutable state, everywhere. Even if you had the person with the expertise, time and motivation to do the work to improve the implementation, you still aren't even guaranteed that the enterprise will succeed. You'll need immense social capital to even convince others that it's actually the right thing to do. Just think of all the new bugs that will be introduced. The angry users. The headache. The added maintenance burden and the pressure from trying to fix the new bugs quickly. It's an absolute nightmare and it's why it doesn't happen, IMO. It's also why starting anew is so much easier and so much more enticing. When you don't have any users, that's when you can really innovate.
  - jerf 8y ago
    "They were written in a time where parallelism wasn't as ubiquitously supported as it is today, and thus, the entire implementation would need to be carefully rethought, refactored and possibly rewritten to grow that benefit. We're talking shared global mutable state, everywhere." I keep a mental list of "things that are effectively impossible to backport onto an existing large source code base", which I really need to keep better track of. (By large, think developer-centuries at a minimum.) But it includes: parallelism/threads/concurrency, adding an external API, transactional correctness, ripping pieces apart and separating them by a network, and transitioning to a fundamentally different data store with different guarantees. All of these have been done, of course, so they aren't literally "impossible". It's just that once you have a source code base with hundreds or thousands of developer-years into them, the amount of effort required and the resulting changes are so extensive that it's not entirely unreasonable to call the result a different code base. You certainly wouldn't end up with anything where you could code new functionality that works against both the old and new codebase without changes, unless you also made that a priority on its own and put a huge additional amount of work into it. (See, for instance, the amount of work PyPy has put into supporting CPython modules. Again, it's possible, barely, but it was on its own terms a very significant amount of work.) So there's a fairly real sense in which you didn't put these features into the code base; you created a new one with a clear parent line of development, but is no longer meaningfully the same. Each of these things, and all the other ones I've forgotten, can be the basis of a new programming language, because at scale it's actually much easier to write a new language like, say, Erlang, and populate it with libraries, and proceed to use it to write tons of multithreaded, Erlang-robust project code than it is to write the same amount of multithreaded, Erlang-robust project code with C. However much work it was to create Erlang libraries, it's dwarfed by orders of magnitude by the amount of Erlang code using it. It's easier to write Rust and populate it with libraries and use it for projects than it is to write the same amount of safe project code with C. Etc.
glangdale 8y ago
Speaking as someone who was the "re-implementee" in some cases (i.e. our mixed C/C++ regex library Hyperscan has had some ideas borrowed for your regex library in Rust) I'd say things aren't 100% simple here. I think there is a strong tendency of people programming in new languages to be overly convinced of the elegance and superiority of their new language as a result of (a) getting to rewrite software without the burden of those fussy, irritating existing users and (b) getting to 'rebuild' software without the rather exhausting process of feeling your way to an algorithm or system that never existed before. There's absolutely nothing wrong with either of these things, but sometimes I think people working in hip new languages tend to mistake the elegance and superiority of their new language for the advantages conferred by a "rework". I bet I could build a pretty elegant regular expression system in C/C++ if I got to start from scratch, too! This isn't a statement about one language being better than another, and I'm aware of how vexed a problem it is to compare new languages with old. If you built a pile of nonesuch systems in your new language people would complain that they can't compare these systems with stuff that they're familiar with. :-) I'd just call for a bit of caution in assessing new languages being pressed into service for old, familiar tasks.
- burntsushi 8y ago
  > There's absolutely nothing wrong with either of these things Sure. But note the OP used the word "irresponsible," so I'd say you're actually disagreeing with the OP, and not me. :-) > I think there is a strong tendency of people programming in new languages to be overly convinced of the elegance and superiority of their new language as a result of (a) getting to rewrite software without the burden of those fussy, irritating existing users and (b) getting to 'rebuild' software without the rather exhausting process of feeling your way to an algorithm or system that never existed before. Yes. But there must also be room for folks to seriously argue the benefits of one tool over another without simultaneously being dismissed as the zealotry of the newly converted, which is a feeling that I am all too familiar with! Overall, I basically agree with you.
  - glangdale 8y ago
    Richard Hamming had a pithy line about this (I believe this is the origin of the phrase "In computer science, we stand on each others feet"): Indeed, one of my major complaints about the computer field is that whereas Newton could say, "If I have seen a little farther than others, it is because I have stood on the shoulders of giants," I am forced to say, "Today we stand on each other's feet." Perhaps the central problem we face in all of computer science is how we are to get to the situation where we build on top of the work of others rather than redoing so much of it in a trivially different way. Science is supposed to be cumulative, not almost endless duplication of the same kind of things.
brightball 8y ago
"It's easier to birth a baby than to resurrect the dead." There's the weight of an existing community that also might make "improvements" more difficult. It's the same reason people leave large companies to go start smaller companies. Could they have stayed and made that company better? Maybe. Maybe not. There's often a lot invested in the status quo.
platz 8y ago
there is always a myth that someone with absolute control of resources could more efficiently re-allocate those efforts. But first you have to obtain absolute top down control in order to do so. Very similar to how command-and-control politics schemes could be in theory more efficient than messy democracies, but those messy democracies, while inefficient, produce variety that guard against the much larger errors of mis-allocation, ineptitude, or corruption. Also the person with the vision of absolute top down control typically selects their vision as the correct one. If only we implemented my vision, since it is the correct one. If we did what the OP suggests we'd be communicating with CORBA across the net, have artifacts such as "car" and "cdr" x10.
- burntsushi 8y ago
  It sounds interesting, but I don't really agree with the idea that we can analogize software development with socio-political organization. There are certainly a lot of interesting similarities, but I would be super careful about drawing any conclusions from it.
  - platz 8y ago
    true, i would offer it more an 'interesting similarities' kind of comment than a call to action.
gonzo41 8y ago
you just gave me a great idea for a new language, Well actually a big library of ASM code that every other language could write wrappers to to get low level gains on all the boilerplate. Then we could all focus on making that really fast.
- amelius 8y ago
  You forget that every language has its own ABI, and runtime quirks. Sending a string to a function works differently in C vs Java vs Python vs ... Sending objects with references to other objects (which may need to be GC'ed) is even more difficult.
  - elcritch 8y ago
    The Truffle VM stuff takes an interesting approach by (afaik) executing the languages getter/setters and then having a common data format (I presume?). What if someone could make a LLVM version of that?
    - mike_hearn 8y ago
      They already did: https://github.com/oracle/graal/tree/master/sulong https://github.com/oracle/graal/tree/master/sulong It's a part of Graal itself now. It runs C, C++, Rust, etc.

pron 8y ago

There are a couple of problems with your perspective: 1. Even if re-implementation of libraries in new languages were free due to an excitement factor, real software is not built this way. New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel. 2. It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work.

pythonaut_16 8y ago
And the flaw with your perspective is that if we all followed this advice, we might have a few highly polished bullet proof C libraries to use, but we'd still be stuck writing C and following all of its paradigms.
- jfoster 8y ago
  That's the interesting part. We're better off because of some but not all new languages having been invented.
  - runevault 8y ago
    Sure, but you have to have the bad attempts to learn things that don't work or we won't end up with the good ones. We fall into a local maximum instead of exploring farther and potentially going higher. Every dead end is another bit of the map explored when it comes to creating expressive languages. Now, is there an argument we'd be better off doing that exploration in a space designed to make DSLs easier than in building entirely new languages? That I dunno but it is worth considering.
- deleted 8y ago
  [deleted]
burntsushi 8y ago
I don't really appreciate the debate tactic of pretending that I made a stronger claim that I actually did. In particular, I didn't claim or imply that re-implementation of things was "free." It is much much easier to demonstrate that claim as ridiculous as opposed to the more measured perspective that I actually expressed. > real software is not built this way This is a bullshit say-nothing phrase. You're using "real software" to presumably denote some dichotomy of software that doesn't exist. If you asked 10 people to define what "real software" actually meant, you'd get 10 different definitions. They might agree on the extremes; but there's a whole mess of crap in the middle that we could endlessly debate about. > New languages become old, and software needs maintenance. So what you get for free (supposing you're right) is not worth very much and not what counts, anyway. Most of the cost and value is in prolonged maintenance, which has to be done for multiple implementations in parallel. This isn't clearly true at all. I mean, if you're going to talk about maintenance, then you also need to consider the impact of improved tools on the costs of maintenance. If a tool makes maintenance easier or less costly, then it's pretty hard to say anything definitive about long term costs. New tools can and do impact the usage of old tools, which in turn can have a direct impact on long term maintenance costs. > It's reasonable to assume that work is never really free. But if we're hypothesizing about alternate realities, you may as well imagine that instead of new languages something else is invented to motivate you to do free work. Absolutely! But that's not the argument I'm countering. I'm specifically countering the notion that new language development can be "irresponsible" because it "could" divert work away from other projects. What I'm saying is that it is not necessarily true at all. There is a more measured argument to be made here, but the OP did not make it. New ideas should be born and die in accordance with the resources of folks willing to keep them alive. We don't need some grand notion of "responsibility" to control the ebb and flow of ideas.
- pron 8y ago
  > You're using "real software" to presumably denote some dichotomy of software that doesn't exist. I think that what I meant was clear from context: software that has a long-term, significant impact. > then you also need to consider the impact of improved tools on the costs of maintenance Sure, but however much maintenance costs, it still needs to be done. It cannot be carried long by a novelty factor. > New ideas should be born and die in accordance with the resources of folks willing to keep them alive. We don't need some grand notion of "responsibility" to control the ebb and flow of ideas. I agree. Anyone should work on whatever they find interesting, and that (normally) doesn't make the choice irresponsible. But I still think that empirically new languages, on average, increasingly cost more and contribute less.

empath75 8y ago

There’s a low-hanging fruit element to new languages as well. Reimplementing is a lot easier than improving, since improving a well understood algorithm requires invention and redoing something only requires research.

burntsushi 8y ago
Definitely. Also, there can be flaws in the actual implementation of older tools as well. I'm going to avoid diving into specifics because it would be a distraction, but I know more than a few older tools that would benefit from parallelism. They were written in a time where parallelism wasn't as ubiquitously supported as it is today, and thus, the entire implementation would need to be carefully rethought, refactored and possibly rewritten to grow that benefit. We're talking shared global mutable state, everywhere. Even if you had the person with the expertise, time and motivation to do the work to improve the implementation, you still aren't even guaranteed that the enterprise will succeed. You'll need immense social capital to even convince others that it's actually the right thing to do. Just think of all the new bugs that will be introduced. The angry users. The headache. The added maintenance burden and the pressure from trying to fix the new bugs quickly. It's an absolute nightmare and it's why it doesn't happen, IMO. It's also why starting anew is so much easier and so much more enticing. When you don't have any users, that's when you can really innovate.
- jerf 8y ago
  "They were written in a time where parallelism wasn't as ubiquitously supported as it is today, and thus, the entire implementation would need to be carefully rethought, refactored and possibly rewritten to grow that benefit. We're talking shared global mutable state, everywhere." I keep a mental list of "things that are effectively impossible to backport onto an existing large source code base", which I really need to keep better track of. (By large, think developer-centuries at a minimum.) But it includes: parallelism/threads/concurrency, adding an external API, transactional correctness, ripping pieces apart and separating them by a network, and transitioning to a fundamentally different data store with different guarantees. All of these have been done, of course, so they aren't literally "impossible". It's just that once you have a source code base with hundreds or thousands of developer-years into them, the amount of effort required and the resulting changes are so extensive that it's not entirely unreasonable to call the result a different code base. You certainly wouldn't end up with anything where you could code new functionality that works against both the old and new codebase without changes, unless you also made that a priority on its own and put a huge additional amount of work into it. (See, for instance, the amount of work PyPy has put into supporting CPython modules. Again, it's possible, barely, but it was on its own terms a very significant amount of work.) So there's a fairly real sense in which you didn't put these features into the code base; you created a new one with a clear parent line of development, but is no longer meaningfully the same. Each of these things, and all the other ones I've forgotten, can be the basis of a new programming language, because at scale it's actually much easier to write a new language like, say, Erlang, and populate it with libraries, and proceed to use it to write tons of multithreaded, Erlang-robust project code than it is to write the same amount of multithreaded, Erlang-robust project code with C. However much work it was to create Erlang libraries, it's dwarfed by orders of magnitude by the amount of Erlang code using it. It's easier to write Rust and populate it with libraries and use it for projects than it is to write the same amount of safe project code with C. Etc.

glangdale 8y ago

Speaking as someone who was the "re-implementee" in some cases (i.e. our mixed C/C++ regex library Hyperscan has had some ideas borrowed for your regex library in Rust) I'd say things aren't 100% simple here. I think there is a strong tendency of people programming in new languages to be overly convinced of the elegance and superiority of their new language as a result of (a) getting to rewrite software without the burden of those fussy, irritating existing users and (b) getting to 'rebuild' software without the rather exhausting process of feeling your way to an algorithm or system that never existed before. There's absolutely nothing wrong with either of these things, but sometimes I think people working in hip new languages tend to mistake the elegance and superiority of their new language for the advantages conferred by a "rework". I bet I could build a pretty elegant regular expression system in C/C++ if I got to start from scratch, too! This isn't a statement about one language being better than another, and I'm aware of how vexed a problem it is to compare new languages with old. If you built a pile of nonesuch systems in your new language people would complain that they can't compare these systems with stuff that they're familiar with. :-) I'd just call for a bit of caution in assessing new languages being pressed into service for old, familiar tasks.

burntsushi 8y ago
> There's absolutely nothing wrong with either of these things Sure. But note the OP used the word "irresponsible," so I'd say you're actually disagreeing with the OP, and not me. :-) > I think there is a strong tendency of people programming in new languages to be overly convinced of the elegance and superiority of their new language as a result of (a) getting to rewrite software without the burden of those fussy, irritating existing users and (b) getting to 'rebuild' software without the rather exhausting process of feeling your way to an algorithm or system that never existed before. Yes. But there must also be room for folks to seriously argue the benefits of one tool over another without simultaneously being dismissed as the zealotry of the newly converted, which is a feeling that I am all too familiar with! Overall, I basically agree with you.
- glangdale 8y ago
  Richard Hamming had a pithy line about this (I believe this is the origin of the phrase "In computer science, we stand on each others feet"): Indeed, one of my major complaints about the computer field is that whereas Newton could say, "If I have seen a little farther than others, it is because I have stood on the shoulders of giants," I am forced to say, "Today we stand on each other's feet." Perhaps the central problem we face in all of computer science is how we are to get to the situation where we build on top of the work of others rather than redoing so much of it in a trivially different way. Science is supposed to be cumulative, not almost endless duplication of the same kind of things.

brightball 8y ago

"It's easier to birth a baby than to resurrect the dead." There's the weight of an existing community that also might make "improvements" more difficult. It's the same reason people leave large companies to go start smaller companies. Could they have stayed and made that company better? Maybe. Maybe not. There's often a lot invested in the status quo.

platz 8y ago

there is always a myth that someone with absolute control of resources could more efficiently re-allocate those efforts. But first you have to obtain absolute top down control in order to do so. Very similar to how command-and-control politics schemes could be in theory more efficient than messy democracies, but those messy democracies, while inefficient, produce variety that guard against the much larger errors of mis-allocation, ineptitude, or corruption. Also the person with the vision of absolute top down control typically selects their vision as the correct one. If only we implemented my vision, since it is the correct one. If we did what the OP suggests we'd be communicating with CORBA across the net, have artifacts such as "car" and "cdr" x10.

burntsushi 8y ago
It sounds interesting, but I don't really agree with the idea that we can analogize software development with socio-political organization. There are certainly a lot of interesting similarities, but I would be super careful about drawing any conclusions from it.
- platz 8y ago
  true, i would offer it more an 'interesting similarities' kind of comment than a call to action.

gonzo41 8y ago

you just gave me a great idea for a new language, Well actually a big library of ASM code that every other language could write wrappers to to get low level gains on all the boilerplate. Then we could all focus on making that really fast.

amelius 8y ago
You forget that every language has its own ABI, and runtime quirks. Sending a string to a function works differently in C vs Java vs Python vs ... Sending objects with references to other objects (which may need to be GC'ed) is even more difficult.
- elcritch 8y ago
  The Truffle VM stuff takes an interesting approach by (afaik) executing the languages getter/setters and then having a common data format (I presume?). What if someone could make a LLVM version of that?
  - mike_hearn 8y ago
    They already did: https://github.com/oracle/graal/tree/master/sulong https://github.com/oracle/graal/tree/master/sulong It's a part of Graal itself now. It runs C, C++, Rust, etc.