4 ms·

That is not any different in Python than it is in Rust, Go or JavaScript. Yes: it's a problem if those different dependency versions shine through via public A

That is not any different in Python than it is in Rust, Go or JavaScript. Yes: it's a problem if those different dependency versions shine through via public APIs. However there are plenty of cases where the dependency you are using in a specific version is isolated within your own internal API. I think if Python were to want to go down this path it should be isolated to explicit migration cases for specific libraries that want to opt themselves into multi-version resolution. I think it would enable the move of pretty core libraries in the ecosystem in backwards incompatible ways in a much smoother way than it is today.

orf 2y ago
The problem with this is exceptions: they easily allow dependencies to escape, be that via uncaught exceptions or wrapped ones. Go and JavaScript have type systems and idioms far more amenable to this kind of thing (interfaces for Go, no real type identity + reliance on structural typing for JS) and rely a lot less on the kind of reflection common in Python (identity, class, etc). I guess there are some use cases for this, I just feel that the lack of ability to enforce visibility combined with the “rock and a hard place” identity trade-off limits the practical usefulness.
- simiones 2y ago
  Exceptions are no different from Go's error types (and in general interface types in any language) from this point of view. If moduleA is doing something like `errors.Is(err, ModuleBError)` on an error that was returned from moduleC which uses a diffferent version of moduleB, you'll get the same issue.
  - orf 2y ago
    That’s interesting - is it common to do this instead of casting to an interface? It seems a lot more impactful with Python due to type equality being core to how exceptions are handled, even if there are similarities.
    - simiones 2y ago
      Well, the most common is of course `if err != nil`, which is unaffected. But in the very rare occasions that someone is actually handling errors in Go, `errors.Is` and `errors.As` are recommended over plain casts since they correctly handle composite exceptions Say a function returns `fmt.Errorf("Error while doing intermediate operation: %w", lowerLevelErr)`, where `lowerLevelErr` is `ModuleBError`. Then, if you do `if _, ok := err.(ModuleBError) {...}`, this will return false; but if you do `if errors.Is(err, ModuleBError)`, you will get the expected true. Regardless, the core problem would be the same: if your code can handle moduleB v1.5 errors but it's receiving moduleB v.17 errors, then it may not be able to handle them. This same thing happens with error values, Exceptions, and in fact any other case of two different implementations returned under the same interface. You even have this problem with C-style integer error codes: say in version 1.5, whenever you try to open a path that is not recognized, you return the int 404. But in 1.7, you return 404 for a missing file, but 407 if it's a missing dir. Any code that is checking for err > 0 will keep working exactly as well, but code which was checking for code 404 to fix missing dir paths is now broken, even though the types are all exactly the same.
- the_mitsuhiko 2y ago
  > The problem with this is exceptions: they easily allow dependencies to escape, be that via uncaught exceptions or wrapped ones. Sure, but that just means your dependency was not really internal. Errors are API too.
kelnos 2y ago
I think the issue is more that in Python you could get confusing runtime failures. In Rust, it will fail to compile if you're trying to mix two different major versions of a dependency like that. I'm fine with the latter, but the former is unacceptable.
- dwattttt 2y ago
  You can have multiple incompatible dependency versions imported by the one create; you have to give them different names when declaring them, but it works fine (I just tested it). It follows the approach of "objects from one version of the library are not compatible with objects of the library" mentioned above, and results in a compile time error (a potentially confusing type error, although the error message might call out that there's multiple library versions involved).
- Macha 2y ago
  It is still painful in Rust, I remember at one stage in a project having a bunch of helper functions to convert structs from one version of nalgebra to another as my game engine (Amethyst at the time, I think?) used one version and the ncollide library used other, and both exposed nalgebra in their public interfaces.

orf 2y ago

The problem with this is exceptions: they easily allow dependencies to escape, be that via uncaught exceptions or wrapped ones. Go and JavaScript have type systems and idioms far more amenable to this kind of thing (interfaces for Go, no real type identity + reliance on structural typing for JS) and rely a lot less on the kind of reflection common in Python (identity, class, etc). I guess there are some use cases for this, I just feel that the lack of ability to enforce visibility combined with the “rock and a hard place” identity trade-off limits the practical usefulness.

simiones 2y ago
Exceptions are no different from Go's error types (and in general interface types in any language) from this point of view. If moduleA is doing something like `errors.Is(err, ModuleBError)` on an error that was returned from moduleC which uses a diffferent version of moduleB, you'll get the same issue.
- orf 2y ago
  That’s interesting - is it common to do this instead of casting to an interface? It seems a lot more impactful with Python due to type equality being core to how exceptions are handled, even if there are similarities.
  - simiones 2y ago
    Well, the most common is of course `if err != nil`, which is unaffected. But in the very rare occasions that someone is actually handling errors in Go, `errors.Is` and `errors.As` are recommended over plain casts since they correctly handle composite exceptions Say a function returns `fmt.Errorf("Error while doing intermediate operation: %w", lowerLevelErr)`, where `lowerLevelErr` is `ModuleBError`. Then, if you do `if _, ok := err.(ModuleBError) {...}`, this will return false; but if you do `if errors.Is(err, ModuleBError)`, you will get the expected true. Regardless, the core problem would be the same: if your code can handle moduleB v1.5 errors but it's receiving moduleB v.17 errors, then it may not be able to handle them. This same thing happens with error values, Exceptions, and in fact any other case of two different implementations returned under the same interface. You even have this problem with C-style integer error codes: say in version 1.5, whenever you try to open a path that is not recognized, you return the int 404. But in 1.7, you return 404 for a missing file, but 407 if it's a missing dir. Any code that is checking for err > 0 will keep working exactly as well, but code which was checking for code 404 to fix missing dir paths is now broken, even though the types are all exactly the same.
the_mitsuhiko 2y ago
> The problem with this is exceptions: they easily allow dependencies to escape, be that via uncaught exceptions or wrapped ones. Sure, but that just means your dependency was not really internal. Errors are API too.

kelnos 2y ago

I think the issue is more that in Python you could get confusing runtime failures. In Rust, it will fail to compile if you're trying to mix two different major versions of a dependency like that. I'm fine with the latter, but the former is unacceptable.

dwattttt 2y ago
You can have multiple incompatible dependency versions imported by the one create; you have to give them different names when declaring them, but it works fine (I just tested it). It follows the approach of "objects from one version of the library are not compatible with objects of the library" mentioned above, and results in a compile time error (a potentially confusing type error, although the error message might call out that there's multiple library versions involved).
Macha 2y ago
It is still painful in Rust, I remember at one stage in a project having a bunch of helper functions to convert structs from one version of nalgebra to another as my game engine (Amethyst at the time, I think?) used one version and the ncollide library used other, and both exposed nalgebra in their public interfaces.