5 ms·

I'm clearly missing something, but why is this so hard? I thought Java made a point of being fully backward compatible, both at the source level and the binary

I'm clearly missing something, but why is this so hard? I thought Java made a point of being fully backward compatible, both at the source level and the binary level (.class/.jar)? Is that true? If so, why is upgrading to a later jdk more work than, well, installing the newer version?

Marazan 6y ago
How Java counts CPUs in a Docker like world has changed multiple times since JDK8. That might not sound too important but it has an effect on GC chosen amd obviously how you might have tuned your thread pools.
sgift 6y ago
Between JDK8 and 9 there was the introduction of the module system, which broke many systems, cause they did somethings they weren't allowed to do in the first place (and which wasn't included in the backwards compatibility guarantees): Access internal JDK classes. There were some compromises to make it still work (e.g. they continued to allow access to some parts of sun.misc.unsafe until there's a replacement with the same performance, cause most of the things in there are pretty fast), but some things just didn't work with the module system and had to be changed. By now, I'd say if you depend on a library that still doesn't work on JDK9+ you should replace it, but depending on your requirements that may not be so easy and so some people are still stuck on JDK8.
Semaphor 6y ago
There are at least some versions that aren’t binary compatible. Because of that we have one system stuck on JDK 7 :/
tandr 6y ago
(not gp) It is about stability, and that magical ephemeral "Something". In software, like JDK or OS, "Something" does change all the time with bug fixes and maybe even just a different builds. Jump from 8 to 11 has a lot of Somethings changed, and transition is not always painless. I have tests that use JSON serialization and deserialization (and then some), and they pass perfectly fine on Java 8 runtime, but fail in exactly same place on Java 11, without recompilation. I have spent some non-zero time trying to get to the bottom, but so far without much luck.
- correct_horse 6y ago
  gson uses reflection to serialize types, including standard library types, which are not guaranteed to have the same fields from one release to another. Presumably the people who write the jdk realized this potential source of bugs and tried to prevent it from happening, meaning fewer ambiguities about what changes are breaking at the expense of breaking gson (and probably others). see: https://github.com/google/gson/issues/1216 https://github.com/google/gson/issues/1216
pvg 6y ago
It's true for the most part, ideally and on average, even between major JDK versions. But there are zillions of production setups out there which are not ideal or average and in turn are made up of not-entirely-ideal or not-entirely-average other software which itself has to go through and verify the upgrade is sane, etc, recursively. Add things like support relationships to all of this and pretty soon you're looking at real work.
pixl97 6y ago
>I thought Java made a point of being fully backward compatible In theory yes. In practice the plugin and scanning systems the software I use has had a multitude of failures between JDK8 and 11.
djsumdog 6y ago
Starting with 9 we saw a lot of functions depreciated for a decade finally removed and other breaking comparability changes. Moving up the Java version system is finally going to stop dependency rot, but there is a lot of old stuff out there, with old dependencies. This might be a Python2/3 moment for Java.
dariusj18 6y ago
Lots of garbage collection changes and jvm flag changes that are not compatible too.
johnc1231 6y ago
Having Apache Spark as a dependency kept us locked into JDK 8 for ages. They've finally got things updated now, but I don't know what their hold up was.
- cmckn 6y ago
  Same here, though it doesn't seem like they've upgraded fully. The Dockerfile[0] for their Kubernetes base images are pinned at 1.8, for example. I bit the bullet and built Spark against newer JDK's myself in the last couple years, but look forward to ditching that. The Hadoop ecosystem has definitely been dragging its feet on JDK bumps. [0] https://github.com/apache/spark/blob/master/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile https://github.com/apache/spark/blob/master/resource-manager...
cesarb 6y ago
A lot (more than you'd expect) of Java projects and libraries depend, directly or indirectly, on the ASM library (https://en.wikipedia.org/wiki/ObjectWeb_ASM https://en.wikipedia.org/wiki/ObjectWeb_ASM), often including a built-in renamed copy. Newer versions of Java add new functionality on the bytecode generated by the Java compiler (in a backwards compatible way, that is, newer versions of Java can fully read bytecode generated by older versions of the Java compiler), and the ASM library has to be updated to understand that new functionality. The Java bytecode verifier has also become more strict, rejecting bytecode generated by older versions of the ASM library and other bytecode generators (things like the "Illegal type in constant pool" error). And in the particular case of Java 9, they decided to deprecate several things, and quickly removed them in Java 11 (that is: with less than a year warning, and no warning if you follow only the long-term versions). That includes a lot of J2EE classes which previously came with the JRE, and now became external dependencies (with the most recent version of these external dependencies sometimes having slightly different behavior).
koreth1 6y ago
Java is better about this than most languages but they still break things sometimes. For example, in addition to the module system, Java 9 changed the implementation of some of its XML rendering libraries and caused meaningful behavior changes. https://github.com/CodeFX-org/java-9-wtf https://github.com/CodeFX-org/java-9-wtf has a list of some of the stuff that broke. That said, 8 -> 9 was the only really painful transition in my experience. Since then the upgrades have been smoother.
deleted 6y ago
[deleted]

Marazan 6y ago

How Java counts CPUs in a Docker like world has changed multiple times since JDK8. That might not sound too important but it has an effect on GC chosen amd obviously how you might have tuned your thread pools.

sgift 6y ago

Between JDK8 and 9 there was the introduction of the module system, which broke many systems, cause they did somethings they weren't allowed to do in the first place (and which wasn't included in the backwards compatibility guarantees): Access internal JDK classes. There were some compromises to make it still work (e.g. they continued to allow access to some parts of sun.misc.unsafe until there's a replacement with the same performance, cause most of the things in there are pretty fast), but some things just didn't work with the module system and had to be changed. By now, I'd say if you depend on a library that still doesn't work on JDK9+ you should replace it, but depending on your requirements that may not be so easy and so some people are still stuck on JDK8.

Semaphor 6y ago

There are at least some versions that aren’t binary compatible. Because of that we have one system stuck on JDK 7 :/

tandr 6y ago

(not gp) It is about stability, and that magical ephemeral "Something". In software, like JDK or OS, "Something" does change all the time with bug fixes and maybe even just a different builds. Jump from 8 to 11 has a lot of Somethings changed, and transition is not always painless. I have tests that use JSON serialization and deserialization (and then some), and they pass perfectly fine on Java 8 runtime, but fail in exactly same place on Java 11, without recompilation. I have spent some non-zero time trying to get to the bottom, but so far without much luck.

correct_horse 6y ago
gson uses reflection to serialize types, including standard library types, which are not guaranteed to have the same fields from one release to another. Presumably the people who write the jdk realized this potential source of bugs and tried to prevent it from happening, meaning fewer ambiguities about what changes are breaking at the expense of breaking gson (and probably others). see: https://github.com/google/gson/issues/1216 https://github.com/google/gson/issues/1216

pvg 6y ago

It's true for the most part, ideally and on average, even between major JDK versions. But there are zillions of production setups out there which are not ideal or average and in turn are made up of not-entirely-ideal or not-entirely-average other software which itself has to go through and verify the upgrade is sane, etc, recursively. Add things like support relationships to all of this and pretty soon you're looking at real work.

pixl97 6y ago

>I thought Java made a point of being fully backward compatible In theory yes. In practice the plugin and scanning systems the software I use has had a multitude of failures between JDK8 and 11.

djsumdog 6y ago

Starting with 9 we saw a lot of functions depreciated for a decade finally removed and other breaking comparability changes. Moving up the Java version system is finally going to stop dependency rot, but there is a lot of old stuff out there, with old dependencies. This might be a Python2/3 moment for Java.

dariusj18 6y ago

Lots of garbage collection changes and jvm flag changes that are not compatible too.

johnc1231 6y ago

Having Apache Spark as a dependency kept us locked into JDK 8 for ages. They've finally got things updated now, but I don't know what their hold up was.

cmckn 6y ago
Same here, though it doesn't seem like they've upgraded fully. The Dockerfile[0] for their Kubernetes base images are pinned at 1.8, for example. I bit the bullet and built Spark against newer JDK's myself in the last couple years, but look forward to ditching that. The Hadoop ecosystem has definitely been dragging its feet on JDK bumps. [0] https://github.com/apache/spark/blob/master/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile https://github.com/apache/spark/blob/master/resource-manager...

cesarb 6y ago

A lot (more than you'd expect) of Java projects and libraries depend, directly or indirectly, on the ASM library (https://en.wikipedia.org/wiki/ObjectWeb_ASM https://en.wikipedia.org/wiki/ObjectWeb_ASM), often including a built-in renamed copy. Newer versions of Java add new functionality on the bytecode generated by the Java compiler (in a backwards compatible way, that is, newer versions of Java can fully read bytecode generated by older versions of the Java compiler), and the ASM library has to be updated to understand that new functionality. The Java bytecode verifier has also become more strict, rejecting bytecode generated by older versions of the ASM library and other bytecode generators (things like the "Illegal type in constant pool" error). And in the particular case of Java 9, they decided to deprecate several things, and quickly removed them in Java 11 (that is: with less than a year warning, and no warning if you follow only the long-term versions). That includes a lot of J2EE classes which previously came with the JRE, and now became external dependencies (with the most recent version of these external dependencies sometimes having slightly different behavior).

koreth1 6y ago

Java is better about this than most languages but they still break things sometimes. For example, in addition to the module system, Java 9 changed the implementation of some of its XML rendering libraries and caused meaningful behavior changes. https://github.com/CodeFX-org/java-9-wtf https://github.com/CodeFX-org/java-9-wtf has a list of some of the stuff that broke. That said, 8 -> 9 was the only really painful transition in my experience. Since then the upgrades have been smoother.

deleted 6y ago

[deleted]