4 ms·
The Java classpath is "flat" by default: you can only have one version of any given class (and therefore de facto only one version of any given library, unless
by lmm 5y ago
The Java classpath is "flat" by default: you can only have one version of any given class (and therefore de facto only one version of any given library, unless they move the library to a different package for each major version, which some libraries do) on the classpath. This has advantages: you avoid the Rust problem of "Foo is incompatible with Foo" because you had two different versions of Foo in your (transitive) dependencies. But it means that if some library you're using uses a utility library internally, that dependency "leaks" and "pollutes" your classpath: you can't use a different version of that utility library yourself.
Modules try to solve this by distinguishing between exported and internal dependencies. The idea is that if your library depends on some other library and uses that as part of its API, that should be an exported dependency (and a program that uses your library has to use the same version of that). But if your library depends on another library just for internal implementation, that gets hidden away from the application that uses your library.
Personally I'm extremely doubtful of this working, because for it to work, library authors would have to specify their metadata correctly. So I suspect it's only going to lead to bigger problems than before, as the classpath gets more complicated and there are more ways to go wrong. But we'll see.
- brabel 5y agoYour analysis focuses on the wrong aspects of the Java module system. It's not meant to solve the problem of having different library versions on the same classpath, and it doesn't solve that at all as you still must ensure only one version of each class exists even if no modules expose those packages (see the split-package problem). The module system is for one thing above all: one more level of visibility, allowing Java developers to specify which packages should be visible to other packages. If you expose a package and that package has public types that expose another library's public types, then you must re-export those packages to make it clear that you have compile-time dependencies on another library, and that's checked so it's not at all hard to specify the appropriate re-exports. The only thing you need to worry about when writing a library is to expose only those packages you want, allowing you to have public classes that are not visible to the outside world unless they are in an public package. Another clarification: Java doesn't really have a single, flat classpath: you can create your own "nested" classpath if you want by instantiating your own class loaders. OSGi works by doing that for each "bundle" (library), which then does allow you to have multiple versions of the same library co-existing peacefully as long as no two packages which use different versions of that library attempt to re-export the library packages to each other (which would fail when you tried to load those bundles into the runtime). You can absolutely get the Rust problem you mentioned in Java if you use your own class loaders, and I've seen that happen several times, even when both libraries are of the same version (a class loaded by a different classloader is NOT the same as the class of the same exact name loaded by another, which is why classloaders are hierarchical and not supposed to load classes already loaded by their parents).
- native_samples 5y ago"It's not meant to solve the problem of having different library versions on the same classpath" It kind of is - there's a single API call that lets you instantiate a module graph with every module mapped to its own classloader. At that point modules work as you'd expect, and package overlaps only create errors if they're both being imported into a single module (where it'd be ambiguous at the language level). As the article explains, this isn't on by default only because (supposedly) some popular frameworks would break due to assumptions about how classloaders are used in practice. And here we hit a sort of weakness in how Java is developed. I've seen this justification given several times over the years for why module isolation isn't on by default. At no point are the frameworks in question named. Which frameworks break when this is on? And why can't I just flip a command line flag to enable or disable it? This compatibility argument is left hanging but no details are given, which is unfortunate - if the frameworks in question were named, maybe they'd fix their assumptions or other people would and classpath conflicts could be fixed.
- mjburgess 5y ago> It kind of is - there's a single API call that lets you instantiate a module graph with every module mapped to its own classloader.... This is what I would expect -- and hence would likely solve most of my issues managing the classpath.
- vlovich123 5y agoCan you elaborate on the Rust problem or give me a pointer to something describing it? I always thought that Rust is perfectly happy to link in conflicting versions of a module into the same binary.
- tsimionescu 5y agoI believe the problem appears when module A returns a value of type B@0.1, and that gets passed on to module C which actually expected a value of type B@0.2. This is a fundamental problem with having multiple versions of the same library used in the same program. The problem is much worse with any kind of dynamic boxing support, since it visibility of the type can't be guaranteed statically. Of course, for many programs, those values are not returned outside the original scope and all works well.
- ampdepolymerase 5y agoCargo is closer to NPM. It does not require SAT resolution. You can have conflicting major versions of the same dependency.
- tsimionescu 5y agoThis was my point as well? You can have different versions of the same library, as long as you do not try to pass around data from one version to the other - which would not work in any language anyway.
- valenterry 5y agoOnly if there is no interaction between them in your code
- cesarb 5y ago> The Java classpath is "flat" by default: you can only have one version of any given class [...] This has advantages: you avoid the Rust problem of "Foo is incompatible with Foo" because you had two different versions of Foo in your (transitive) dependencies. The Java classpath is "flat" only if you use a single classloader. If you use multiple classloaders (for instance, with a plugin system where each plugin is loaded on its own classloader, to allow it to be dynamically loaded and unloaded), you can have multiple versions of any given class. At work, we have more than once had that "Foo is incompatible with Foo" problem (made more maddening because the "Foo is incompatible with Foo" error messages don't mention which classloader each "Foo" came from).
- sebazzz 5y agoThat's correct. This is especially noticable with application servers like Jboss that deliver their own libraries in the class path. Needing a slightly higher version of Hibernate is... interesting.