5 ms·
Something Java and the Java ecosystem is terrible at. I've had project where we needed all of 7,8,9 and 10 [0] for the data stack: Hadoop, Scala, Spark, HDFS, z
by buzzkillington 7y ago
Something Java and the Java ecosystem is terrible at. I've had project where we needed all of 7,8,9 and 10 [0] for the data stack: Hadoop, Scala, Spark, HDFS, zookeeper, pyspark. Good luck setting that up on a single machine, containers all the way. The java path nightmares I still have. I'm probably one of the few people in the word who know this much about the JVM without ever having written any Java.
[0] We should have also been following 11 but just didn't have the resources to even think about doing it.
- apta 7y ago> I've had project where we needed all of 7,8,9 and 10 [0] for the data stack: Hadoop, Scala, Spark, HDFS, zookeeper, pyspark. I suspect that a big reason was due to using Scala, which is notorious for not maintaining binary backward compatibility across minor releases.
- paulddraper 7y agoTo be fair, most languages don't maintain backward binary compatibility. In fact, this is so common that some package managers don't even bother distributing binaries: go modules, cargo, conan, etc.
- raquo 7y agoScala's version is epoch.major.minor, not major.minor.patch. I though 2.12.(x+1) is generally binary compatible with 2.12.x. Or at least I personally never ran into that issue with Scala itself.
- lmm 7y agoScala breaks binary compatibility approximately every 2-3 years, which is fast by JVM standards but slow compared to most programming languages (e.g. Python releases an incompatible version every 1-2 years). You can blame Scala for at most one incompatibility in that long list; older versions of Scala are incompatible with Java 9+ (this is true for many JVM frameworks too, as Java 9 made breaking changes).