3 ms·

Let alone lines of code would be even more when it's written in C++.

• tormeh 5y ago
  A lot more. Scala, the Akka ecosystem, and the Hadoop ecosystem is uniquely suited to Spark's problem domain. And Scala is, for both better and worse, an extremely compact language too. I would expect about 5x more lines of code just from that.
  • mumblemumble 5y ago
    One interesting thing here is that there are people who know some real numbers, and not just hypotheticals, but they probably can't talk about it publicly. Google has their own distributed compute engine, written in C++. Everyone I know who's worked with it says it's much nicer than Spark, and much more performant. I wouldn't necessarily just assume that it's 5x more code. I've spent a fair bit of time digging around in the Spark code, and it's not clear to me that it's as efficiently designed or cleanly implemented as one might like. And, while I can't speak to the influence of Google's style guide, I don't personally feel like modern C++ is 5x as verbose as modern Scala. A bit more, maybe? But it's not like Scala (2, at least) doesn't have its own bizarre sources of unnecessary verbosity. It mostly looks great by comparison to Java.