3 ms·

It's a problem because you can only have one version of any given package in your dependency tree. You can't have `foo 2` and `foo 3` in your dependency tree.

It's a problem because you can only have one version of any given package in your dependency tree. You can't have `foo 2` and `foo 3` in your dependency tree. That does seem to be the fundamental problem with the Python model of dependency management. If your dependencies have transitive dependencies of their own but your dependency model is a tree and everything is clearly namespaced/versioned, you might end up with multiple versions of the same package installed, but at least they won’t conflict. If your dependency model is flat but each dependency bakes in its own transitive dependencies so they’re hidden from the rest of the system, for example via static linking, again you might end up with multiple versions of the same package (or some part of it) installed, but again they won’t conflict. But if your dependency model is flat and each dependency can require specific versions of its transitive dependencies to be installed as peers, you fundamentally can’t avoid the potential for unresolvable conflicts. A pragmatic improvement in the third case is, as others have suggested, to replace the SemVer-following mypackage 1.x.y and mypackage 2.x.y with separate top-level packages mypackage1 x.y and mypackage2 x.y. Now you have reintroduced namespaces and you can install mypackage1 and mypackage2 together as peers without conflict. Moreover, if increasing x and y faithfully represent minor and point releases, always using the latest versions of mypackage1 and mypackage2 should normally satisfy any other packages that depend on them, however many there are. Of course it doesn’t always work like that in practice. However, at least the problem is now reduced to manually adjusting versions to resolve conflicts where a package didn’t match its versions to its behaviour properly and/or Hyrum’s Law is relevant, which is probably much less work than before.

int_19h 2y ago
As the article explains, this is precisely why the social expectations around Python package versioning are very different from JS package version (i.e. you can't just break things willy nilly even in major releases and cite semver as justification). That aside, note the obvious problems here for any language that uses nominal typing - like, say, Python. Since types from dependencies can often surface in one's public API, having a tree of dependencies means that many libraries will end up referring to different (and thus ipso facto incompatible) versions of the same type.
- Chris_Newton 2y ago
  social expectations around Python package versioning are very different from JS package version If anything, I’d say in my experience the Python community tends to be more willing to make big changes. After all, Python itself famously did so with the 2 to 3 transition, and to some extent we’re seeing a second round of big changes even now as optional typing spreads through the ecosystem. Admittedly, the difference could also be because so few packages in JS world seem to last long enough for multiple major versions to become an issue. The Python ecosystem seems more willing to settle on a small number of de facto standard libraries for common tasks. Since types from dependencies can often surface in one's public API, having a tree of dependencies means that many libraries will end up referring to different (and thus ipso facto incompatible) versions of the same type. Leaving aside the questionable practice of exposing details of internal dependencies directly through one’s own public interface, I don’t see how this is any different to any other potential naming conflict. Whatever dependency model you pick, you’re always going to have the possibility that two dependencies use the same name as part of their interface, and in Python you’re always going to have to disambiguate explicitly if you want to import both in the same place. However, once you’ve done so, there is no longer any naming clash to leak through your own interface either.
  - int_19h 2y ago
    > After all, Python itself famously did so with the 2 to 3 transition That transition has been so traumatic for the whole ecosystem that, if anything, it became an abject lesson as to why you don't do stuff like that. "Never again" is the current position of PSF wrt any hypothetical future Python 3 -> 4 transition. Major Python libraries pretty much never just remove things over the course of a single major release. Things get officially announced first, then deprecated for at least one release cycle but often longer (which is communicated via DeprecationWarning etc), then finally retired. > Leaving aside the questionable practice of exposing details of internal dependencies directly through one’s own public interface Not all dependencies are internal. If library A exposes type X, and library B exposes type Y that by design extends X (so that instances of Y can be passed anywhere X is expected), that is very intentionally public. Now imagine that library C exposes type Z that also by design extends X. If B and C both get their copy of A, then there are two identical types X that are not type-compatible. Now suppose we have the app that depends on both B and C. Its author wants to write a generic function F that accepts an instance of X (or a subtype) and does something with it. How do they write a type signature for F such that it can accept both Y and Z?

int_19h 2y ago

As the article explains, this is precisely why the social expectations around Python package versioning are very different from JS package version (i.e. you can't just break things willy nilly even in major releases and cite semver as justification). That aside, note the obvious problems here for any language that uses nominal typing - like, say, Python. Since types from dependencies can often surface in one's public API, having a tree of dependencies means that many libraries will end up referring to different (and thus ipso facto incompatible) versions of the same type.

Chris_Newton 2y ago
social expectations around Python package versioning are very different from JS package version If anything, I’d say in my experience the Python community tends to be more willing to make big changes. After all, Python itself famously did so with the 2 to 3 transition, and to some extent we’re seeing a second round of big changes even now as optional typing spreads through the ecosystem. Admittedly, the difference could also be because so few packages in JS world seem to last long enough for multiple major versions to become an issue. The Python ecosystem seems more willing to settle on a small number of de facto standard libraries for common tasks. Since types from dependencies can often surface in one's public API, having a tree of dependencies means that many libraries will end up referring to different (and thus ipso facto incompatible) versions of the same type. Leaving aside the questionable practice of exposing details of internal dependencies directly through one’s own public interface, I don’t see how this is any different to any other potential naming conflict. Whatever dependency model you pick, you’re always going to have the possibility that two dependencies use the same name as part of their interface, and in Python you’re always going to have to disambiguate explicitly if you want to import both in the same place. However, once you’ve done so, there is no longer any naming clash to leak through your own interface either.
- int_19h 2y ago
  > After all, Python itself famously did so with the 2 to 3 transition That transition has been so traumatic for the whole ecosystem that, if anything, it became an abject lesson as to why you don't do stuff like that. "Never again" is the current position of PSF wrt any hypothetical future Python 3 -> 4 transition. Major Python libraries pretty much never just remove things over the course of a single major release. Things get officially announced first, then deprecated for at least one release cycle but often longer (which is communicated via DeprecationWarning etc), then finally retired. > Leaving aside the questionable practice of exposing details of internal dependencies directly through one’s own public interface Not all dependencies are internal. If library A exposes type X, and library B exposes type Y that by design extends X (so that instances of Y can be passed anywhere X is expected), that is very intentionally public. Now imagine that library C exposes type Z that also by design extends X. If B and C both get their copy of A, then there are two identical types X that are not type-compatible. Now suppose we have the app that depends on both B and C. Its author wants to write a generic function F that accepts an instance of X (or a subtype) and does something with it. How do they write a type signature for F such that it can accept both Y and Z?