5 ms·

A really thought-provoking argument. I suppose that when it comes to choosing "the right tool" the important thing is to optimize globally across the problems

A really thought-provoking argument. I suppose that when it comes to choosing "the right tool" the important thing is to optimize globally across the problems that your organization will be solving. But if you want to be able to optimize both globally and locally, well that's just an engineering problem! Two examples spring to mind: 1. Lisp. Lisp's metaprogramming facilities allow you to build the language into the specialized tools that you need, while still having the common runtime and base language. 2. Unix. There a hundreds (possibly even thousands) of little programs running on my Linux machine written in probably dozens of languages. For the most part, I have no idea (nor do I care) what language they are written in. That's because the Unix model puts a strong emphasis on the protocols for communicating between programs.

jeffdavis 7y ago
Lisp is an interesting point. I guess the argunent is that it's the right tool for every job? Unix is a reasonable example of a polyglot environment, but at a high cost. Lots of serialization/deserialization through text. That has a high cost in terms of bugs, complexity, lines of code, and inefficiency.
- lliamander 7y ago
  > Lisp is an interesting point. I guess the argunent is that it's the right tool for every job? Yup, that's more or less the argument, though the argument need not only apply to lisp. > Unix is a reasonable example of a polyglot environment, but at a high cost. I'd rather say that Unix is an example of how to effectively drive down the cost of a polyglot environment to a point where it is outweighed by the benefits. Now, of course the cost is still there, but I would argue that beyond a certain scale you cannot optimize globally on a single runtime (perhaps not even with something like lisp) and so the cost of global consistency is outweighed by cost of being unable to optimize locally. To put it more succintly, you could not build a system as complex as Unix in a single language[1] [1] Although Ala Kay's work at VPRI suggests you can, if you choose a sufficiently expressive base language. But even that may have it's limits.
continuational 7y ago
Those programs communicate by serialization and deserialization, usually in bespoke, poorly documented data formats. Unix doesn't put a strong emphasis on protocols. It just says "everything is text, except when it's not". It's not very helpful.
- lliamander 7y ago
  > Unix doesn't put a strong emphasis on protocols. It just says "everything is text, except when it's not". It's not very helpful. OK, I probably misspoke by claiming it put an emphasis in protocols, but I think it's fair to say that Unix does emphasize component integration by sharing data instead of trying to integrate through a common runtime. I do think that having everything be (mostly) text is helpful though. It's true that the data formats are often poorly specified, but that is compensated for by the fact text is a format that has a very rich set of tooling. text editors, regular expressions, parser-generators, etc. all make it possible to capture, analyze, and manipulate the text data exchanged. Perhaps a better example of protocol-based integration would be the internet. It has it's flaws as well, but it's also enabled collective engineering projects on a vast scale.
- effie 7y ago
  Modern unix gives you mostly low-level mechanisms, not inter-application protocols. It is better to leave it up to applications/administrators to figure out the best protocol (policy) for their use case, to make the best use of those mechanisms. Those mechanisms aren't textual, they are byte-sequence based. It turns out that many text-based protocols are more popular than binary/object alternatives. The penalty for textual redundancy is negligible/acceptable for many of them. Where not, Unix allows people to use unix/create new mechanisms to implement specialized protocols (rpc, protobuffers,...).
adev_ 7y ago
You got it yes. And very few Software Engineer understand that unfortunately. The reason C is so successful is named ABI. C (and C++ to some extend) can be used to produce library that can be used from any higher level language in existence. And this is strictly required in the heterogeneous world we live in. If you have to write a system components, long lifetime, implementing a protocol / database / format / any-low-level-staff: You do not have choice, safety or not. It is going to be C or C++, because it is the only thing reusable in most other environment. Python has pybind11, Java has JNI, Lua operate beautifully with C/C++, Node is itself C++, go has cgo, Ruby is in C, any proper programming language can interface with C or C++. They are the only languages allowing that, and that is why they are still alive, very well alive, even if they are unsafe. As long as new language authors will be more obsessed about supporting new fancy feature instead of providing a well defined ABI (C compatible). The situation will not change. Rust might be the new comer in this area that has its chance. But for the time being, it is still too young: you have to map every Rust API to a C one manually to make it exportable.

jeffdavis 7y ago

Lisp is an interesting point. I guess the argunent is that it's the right tool for every job? Unix is a reasonable example of a polyglot environment, but at a high cost. Lots of serialization/deserialization through text. That has a high cost in terms of bugs, complexity, lines of code, and inefficiency.

lliamander 7y ago
> Lisp is an interesting point. I guess the argunent is that it's the right tool for every job? Yup, that's more or less the argument, though the argument need not only apply to lisp. > Unix is a reasonable example of a polyglot environment, but at a high cost. I'd rather say that Unix is an example of how to effectively drive down the cost of a polyglot environment to a point where it is outweighed by the benefits. Now, of course the cost is still there, but I would argue that beyond a certain scale you cannot optimize globally on a single runtime (perhaps not even with something like lisp) and so the cost of global consistency is outweighed by cost of being unable to optimize locally. To put it more succintly, you could not build a system as complex as Unix in a single language[1] [1] Although Ala Kay's work at VPRI suggests you can, if you choose a sufficiently expressive base language. But even that may have it's limits.

continuational 7y ago

Those programs communicate by serialization and deserialization, usually in bespoke, poorly documented data formats. Unix doesn't put a strong emphasis on protocols. It just says "everything is text, except when it's not". It's not very helpful.

lliamander 7y ago
> Unix doesn't put a strong emphasis on protocols. It just says "everything is text, except when it's not". It's not very helpful. OK, I probably misspoke by claiming it put an emphasis in protocols, but I think it's fair to say that Unix does emphasize component integration by sharing data instead of trying to integrate through a common runtime. I do think that having everything be (mostly) text is helpful though. It's true that the data formats are often poorly specified, but that is compensated for by the fact text is a format that has a very rich set of tooling. text editors, regular expressions, parser-generators, etc. all make it possible to capture, analyze, and manipulate the text data exchanged. Perhaps a better example of protocol-based integration would be the internet. It has it's flaws as well, but it's also enabled collective engineering projects on a vast scale.
effie 7y ago
Modern unix gives you mostly low-level mechanisms, not inter-application protocols. It is better to leave it up to applications/administrators to figure out the best protocol (policy) for their use case, to make the best use of those mechanisms. Those mechanisms aren't textual, they are byte-sequence based. It turns out that many text-based protocols are more popular than binary/object alternatives. The penalty for textual redundancy is negligible/acceptable for many of them. Where not, Unix allows people to use unix/create new mechanisms to implement specialized protocols (rpc, protobuffers,...).

adev_ 7y ago

You got it yes. And very few Software Engineer understand that unfortunately. The reason C is so successful is named ABI. C (and C++ to some extend) can be used to produce library that can be used from any higher level language in existence. And this is strictly required in the heterogeneous world we live in. If you have to write a system components, long lifetime, implementing a protocol / database / format / any-low-level-staff: You do not have choice, safety or not. It is going to be C or C++, because it is the only thing reusable in most other environment. Python has pybind11, Java has JNI, Lua operate beautifully with C/C++, Node is itself C++, go has cgo, Ruby is in C, any proper programming language can interface with C or C++. They are the only languages allowing that, and that is why they are still alive, very well alive, even if they are unsafe. As long as new language authors will be more obsessed about supporting new fancy feature instead of providing a well defined ABI (C compatible). The situation will not change. Rust might be the new comer in this area that has its chance. But for the time being, it is still too young: you have to map every Rust API to a C one manually to make it exportable.