4 ms·
The use or adherence to semver isn't the problem here. As you say, if a package follows semver, it's easy enough for the package managers to automatically updat
by kibwen 5mo ago
The use or adherence to semver isn't the problem here. As you say, if a package follows semver, it's easy enough for the package managers to automatically update to newer compatible versions. The problem is when you want to have two different incompatible versions of the same package `foo` in the same program, because then you have to figure out what `import foo` means. You might say "just don't do that", but that package could be an indirect dependency of several of your direct dependencies. Some languages handle this natively, e.g. in Rust it just works if you have multiple versions of the same library in different parts of your dependency tree (and you'll get a compilation error if you try to pass a type from one version into a function of an incompatible version). But Python does not handle this use case very well.
- skeledrew 5mo ago> Python does not handle this use case very well I solved this issue a few months ago. Created a tool that essentially allows the use of multiple envs at once, with their own versions of packages at any level.
- mroche 5mo agoThis sounds... not possible for the core problem of how Python handles dependency resolution during the life of an application? How are you setting things up so that the following scenario is valid? program ├── dependency_a │ └── dependency_c (1.0.0) └── dependency_b └── dependency_c (2.0.0) Otherwise, you've created a magic layer hack to enable multi-version dependency chains in a mono-version dependency chain language.
- yorwba 5mo agoPython's import system is extensible: https://docs.python.org/3/reference/import.html#import-hooks https://docs.python.org/3/reference/import.html#import-hooks It might be possible to create a custom finder that will return 1.0.0 when running "import dependency_c" in dependency_a but 2.0.0 for the same import statement in dependency_b. You'll need to work around the module cache in sys.modules, though. And good luck trying this on a package that also hooks the import system...
- skeledrew 5mo agoYep that's pretty much how I do it, along with RPyC[0] for connection. [0] https://rpyc.readthedocs.io/en/latest/ https://rpyc.readthedocs.io/en/latest/
- skeledrew 5mo agoI'll try to get around to a Show HN some time for it, but it's essentially as sibling reply states.
- arcatek 5mo agoCurious how you did this; I looked into that couple of months ago but even with custom hooks the Python injection points seemed to limited due to the internal resolution cache.
- skeledrew 5mo agoIts just objects accessed via RPyC[0] and exposed with import hooks. Thinking of a Show HN but there repo isn't really publish-ready. [0] https://rpyc.readthedocs.io/en/latest/ https://rpyc.readthedocs.io/en/latest/
- __mharrison__ 5mo agoI'm curious. Do you have real world examples of when you want to do this?
- gbear605 5mo agoThis is Java, but recently I had a case where one library depended on a version of an Apache Commons library, and another library depended on a different version of the same Apache Commons library, and neither version worked with both libraries. In my case, I was able to upgrade one of them to a newer version so that I could use just one Apache Commons version, but I got lucky there.
- deathanatos 5mo agoThe monolith I work on has this dependency chain: monolith -> openai monolith -> langchain-openai -> openai openai, thus, is both a direct and indirect dependency. langchain-openai recently had a vulnerability, and the patch fix is only after a major upgrade to openai. Thus, to upgrade langchain-openai here, I also need to upgrade monolith's use of openai. (From v1 to v2.)