4 ms·

> what's left? - Erlang (this fits your criteria at least as Java...) - OCaml (concurrency isn't the best) - Haskell (also fits your criteria very well) - D

by PeCaN 10y ago

> what's left? - Erlang (this fits your criteria at least as Java...) - OCaml (concurrency isn't the best) - Haskell (also fits your criteria very well) - D (easy to use C libraries and sometimes C++) - Rust (not GCed, but why is GCed a requirement?) - Mercury (admittedly pretty obscure, but using C libraries is easy enough and it fits everything else) Java only seems like a good choice if you ignore all the languages that are a better choice than Java. IMO Erlang seems like the choice here (or Elixir if you don't like Erlang syntax).

mbreese 10y ago
Let's also say that you want to be able to hire programmers to write and maintain it. And it would be nice to make it open to many more people so they could find and fix bugs. After that, what's left? At the moment, none of the above languages have any significant talent pool to draw from. If you're operating at "Twitter-scale" (I guess that's now a thing), you also need to be able to spin up a team of people that can get to work quickly without having to learn a language as they go. That eliminates a great number of languages just for logistical reasons. Rust might get there (and I hope it does!), but it's not there yet. Maybe Twitter already has a significant number of skilled Java engineers, so they decided to use what they already knew really well. There's nothing wrong with that. Choosing any of the other languages you listed would have been a far riskier strategy.
flippant 10y ago
- Erlang/Elixir - OP mentioned performance. Erlang is terrible for CPU-bound tasks. - Haskell - OP said non-esoteric. - Rust - There's a significant ramp up before you learn to deal with the compiler and lifetimes.
- PeCaN 10y ago
  > Erlang Erlang's CPU performance is not bad IME (though admittedly I can't think of anything I've implemented in both Erlang and Java to compare). But that's beside the point; you pick Erlang for networked and I/O bound tasks. Twitter mentioned running Heron on several hundred machines—at that scale, single-thread performance starts to matter less and Erlang's scaling efficiency closes the gap. > Haskell Haskell may have been esoteric in 2005, but it's hardly so anymore. (Also, I mention Mercury, and Haskell is the language you decide is too esoteric‽) > Rust So you're saying they wouldn't use it because they didn't know it. I suspect that's the real reason they chose Java—not because it is necessarily the best language suited for the task, but because it's the best of the languages Heron's developers knew.
  - asdf1234 10y ago
    > Erlang's CPU performance is not bad IME Erlang is very slow when it comes to CPU bound tasks. It's nowhere near Java for tasks like this.
  - galistoca 10y ago
    What is the definition of "best" anyway? If you really just think about pure performance nothing can beat a binary code probably. When you say "best" it probably means most productive in the short/long term. In that case it's not about what the language is good at. After all it's humans who use the language. Just like how esperanto is such an awesome concept but nobody uses it and that makes it a useless language, the more people use a language the more valuable it becomes. And yeah Haskell is esoteric. I am confident that I know way many more programming languages than an average programmer but I don't program in Haskell. Try picking 10 random programmers and ask them if they can program in Haskell, at best you'll find 1 (actually that's even being too optimistic)
zjaffee 10y ago
None of these languages even come close to the level of library support and popularity as Java. Additionally it becomes a people problem when you use a language people aren't as familiar with leading to people building connectors which don't perform as well as using the original libraries. With Java, you can directly import a lot more, especially since the vast majority of OSS in this ecosystem is written in a JVM language.
hansjorg 10y ago
I think you're underestimating the value of the enormous Java/JVM ecosystem. That said, I'm hoping to deploy my first Rust project to production soon. It's come a long way for being such a young language.
- PeCaN 10y ago
  > I think you're underestimating the value of the enormous Java/JVM ecosystem. You could be right. For a number of the languages I mentioned I banked on really good FFIs allowing easy usage of C libraries. Perhaps in some domains, including stream processing, there are more and better Java or C++ libraries for them to build on.
- nstart 10y ago
  Hey.. I'm just super curious to learn a lot about the Rust ecosystem. Would you be up for sharing any details of what your rust project might be? Thanks!
pron 10y ago
Aside from Erlang which is far too slow (most Erlang apps use C for the data plane) the others are far too risky choices in terms of hiring, future support and general maintainability. Also there are far too many unknowns about them. Rust shows a lot of promise and may very well be suitable for this kind of things, but it may still be too young for an important infrastructure project at Twitter. It is also harder to program than in Java, although for an infrastructure project such as this, this may not be a deal breaker. Another reason to run on the JVM is that often your applications run on the JVM, and the JIT -- which is getting better and better, and will get incredibly good in Java 9 -- can really do cool stuff when it optimizes across the app/infrastructure line. I've just seen a paper[1] showing 3x performance boost when rewriting parts of SQLite in Python, and letting the JIT optimize the DB together with the app. [1]: http://arxiv.org/abs/1512.03207 http://arxiv.org/abs/1512.03207
- hyperpape 10y ago
  Is there any kind of overview of the JIT improvements coming in Java 9, or do you have to follow the relevant mailing lists/commits?
  - pron 10y ago
    I'm referring to Graal, which would be available in Java 9 as an alternative JIT to HotSpot's C2 compiler: https://wiki.openjdk.java.net/display/Graal/Publications+and+Presentations https://wiki.openjdk.java.net/display/Graal/Publications+and...
- technion 10y ago
  Aside from Erlang which is far too slow (most Erlang apps use C for the data plane) I'm going to call "citation needed" on that. Riak and EJabberd have a reputation for scale and performance, and the only C I can see referenced is where existing libraries like Zlib get used. Edit: You are still right. The hiring issues are relevant, and apply to Erlang also.
010a 10y ago
Finding developers is hard. Finding developers who know those languages to support your software is near impossible.
abiox 10y ago
java isn't that bad. there's a lot of crappy code-culture that comes from the uber-enterprisey arena, but in more reasonable hands it's quite capable. it's also worth nothing that Java comes with the JVM and Hotspot JIT, which earn it a bit of swagger.
lpw25 10y ago
> OCaml (concurrency isn't the best) Pet peeve: OCaml's concurrency is pretty good, it's parallelism that it struggles with.

by PeCaN 10y ago

mbreese 10y ago

Let's also say that you want to be able to hire programmers to write and maintain it. And it would be nice to make it open to many more people so they could find and fix bugs. After that, what's left? At the moment, none of the above languages have any significant talent pool to draw from. If you're operating at "Twitter-scale" (I guess that's now a thing), you also need to be able to spin up a team of people that can get to work quickly without having to learn a language as they go. That eliminates a great number of languages just for logistical reasons. Rust might get there (and I hope it does!), but it's not there yet. Maybe Twitter already has a significant number of skilled Java engineers, so they decided to use what they already knew really well. There's nothing wrong with that. Choosing any of the other languages you listed would have been a far riskier strategy.

flippant 10y ago

- Erlang/Elixir - OP mentioned performance. Erlang is terrible for CPU-bound tasks. - Haskell - OP said non-esoteric. - Rust - There's a significant ramp up before you learn to deal with the compiler and lifetimes.

PeCaN 10y ago
> Erlang Erlang's CPU performance is not bad IME (though admittedly I can't think of anything I've implemented in both Erlang and Java to compare). But that's beside the point; you pick Erlang for networked and I/O bound tasks. Twitter mentioned running Heron on several hundred machines—at that scale, single-thread performance starts to matter less and Erlang's scaling efficiency closes the gap. > Haskell Haskell may have been esoteric in 2005, but it's hardly so anymore. (Also, I mention Mercury, and Haskell is the language you decide is too esoteric‽) > Rust So you're saying they wouldn't use it because they didn't know it. I suspect that's the real reason they chose Java—not because it is necessarily the best language suited for the task, but because it's the best of the languages Heron's developers knew.
- asdf1234 10y ago
  > Erlang's CPU performance is not bad IME Erlang is very slow when it comes to CPU bound tasks. It's nowhere near Java for tasks like this.
- galistoca 10y ago
  What is the definition of "best" anyway? If you really just think about pure performance nothing can beat a binary code probably. When you say "best" it probably means most productive in the short/long term. In that case it's not about what the language is good at. After all it's humans who use the language. Just like how esperanto is such an awesome concept but nobody uses it and that makes it a useless language, the more people use a language the more valuable it becomes. And yeah Haskell is esoteric. I am confident that I know way many more programming languages than an average programmer but I don't program in Haskell. Try picking 10 random programmers and ask them if they can program in Haskell, at best you'll find 1 (actually that's even being too optimistic)

zjaffee 10y ago

None of these languages even come close to the level of library support and popularity as Java. Additionally it becomes a people problem when you use a language people aren't as familiar with leading to people building connectors which don't perform as well as using the original libraries. With Java, you can directly import a lot more, especially since the vast majority of OSS in this ecosystem is written in a JVM language.

hansjorg 10y ago

I think you're underestimating the value of the enormous Java/JVM ecosystem. That said, I'm hoping to deploy my first Rust project to production soon. It's come a long way for being such a young language.

PeCaN 10y ago
> I think you're underestimating the value of the enormous Java/JVM ecosystem. You could be right. For a number of the languages I mentioned I banked on really good FFIs allowing easy usage of C libraries. Perhaps in some domains, including stream processing, there are more and better Java or C++ libraries for them to build on.
nstart 10y ago
Hey.. I'm just super curious to learn a lot about the Rust ecosystem. Would you be up for sharing any details of what your rust project might be? Thanks!

pron 10y ago

Aside from Erlang which is far too slow (most Erlang apps use C for the data plane) the others are far too risky choices in terms of hiring, future support and general maintainability. Also there are far too many unknowns about them. Rust shows a lot of promise and may very well be suitable for this kind of things, but it may still be too young for an important infrastructure project at Twitter. It is also harder to program than in Java, although for an infrastructure project such as this, this may not be a deal breaker. Another reason to run on the JVM is that often your applications run on the JVM, and the JIT -- which is getting better and better, and will get incredibly good in Java 9 -- can really do cool stuff when it optimizes across the app/infrastructure line. I've just seen a paper[1] showing 3x performance boost when rewriting parts of SQLite in Python, and letting the JIT optimize the DB together with the app. [1]: http://arxiv.org/abs/1512.03207 http://arxiv.org/abs/1512.03207

hyperpape 10y ago
Is there any kind of overview of the JIT improvements coming in Java 9, or do you have to follow the relevant mailing lists/commits?
- pron 10y ago
  I'm referring to Graal, which would be available in Java 9 as an alternative JIT to HotSpot's C2 compiler: https://wiki.openjdk.java.net/display/Graal/Publications+and+Presentations https://wiki.openjdk.java.net/display/Graal/Publications+and...
technion 10y ago
Aside from Erlang which is far too slow (most Erlang apps use C for the data plane) I'm going to call "citation needed" on that. Riak and EJabberd have a reputation for scale and performance, and the only C I can see referenced is where existing libraries like Zlib get used. Edit: You are still right. The hiring issues are relevant, and apply to Erlang also.

010a 10y ago

Finding developers is hard. Finding developers who know those languages to support your software is near impossible.

abiox 10y ago

java isn't that bad. there's a lot of crappy code-culture that comes from the uber-enterprisey arena, but in more reasonable hands it's quite capable. it's also worth nothing that Java comes with the JVM and Hotspot JIT, which earn it a bit of swagger.

lpw25 10y ago

> OCaml (concurrency isn't the best) Pet peeve: OCaml's concurrency is pretty good, it's parallelism that it struggles with.