5 ms·

Should this be facilitated? Should this work really be done? I’m thinking not. If I have a special case and I need to do this today, I’m not blocked from doing

Should this be facilitated? Should this work really be done? I’m thinking not. If I have a special case and I need to do this today, I’m not blocked from doing so (I’ll vendor one of the dependencies and change the name) - certainly a pain to do, especially if I have to go change imports in a c module as part of the dep but achievable and not a blocker for a source available dependency. However, if this becomes easily possible, well why shouldn’t I use it? The net result is MORE complexity in python packaging. More overheads for infra tools to accomadate.

baq 2y ago
You’ve never found yourself in a dependency resolution situation where there are no solutions to requirements. You need multiple versions of the same package in such cases. The alternative is just cheating: ignore requirements, install packages and hope for the best. Alas hope is not a process.
- dikei 2y ago
  The usual way is to put conflicting versions in optional-dependencies, and then build one target for each conflicting set of deps. That'd work fine if the code path of one target doesn't touch the others, which is often the case. You'd obviously need to have tests for both targets, possibly using a flexible test runner like `nox` to setup separate test env for each target.
- theelous3 2y ago
  I've never found myself in this situation where it can't be solved by being throughtful and taking the time to improve my code.
  - leetrout 2y ago
    These issues crop up in dependencies more than your code. Then you have to vendor one of the deps and edit it (and hopefully ship that back upstream and hope the maintainer will merge it).
- zokier 2y ago
  No, the alternative is to just fix the code so that the conflict is removed. This is why open source is so powerful, you are empowered to fix stuff instead of piling up tower of workarounds.
  - HelloNurse 2y ago
    You can fix your code, but indirect dependencies (you use A and B, both depend on C, but different versions of C) cannot be handled well. In C, C++, maybe Java, you would at least be able to link A and B with their own private copies of C to avoid conflicts reliably with standard mechanisms rather than unreliably with clever magical tools.
    - Too 2y ago
      You missed the point. If A, B and C are all open source you can actually fix them too. Just send a PR. Most projects are open to ensuring compatibility moves forward.
      - HelloNurse 2y ago
        Just send a PR? What are you talking about? A project that requires an old version of a dependency either has technically valid reasons, and is unlikely to be upgraded just because one more user asks nicely, or is maintained at a slow pace and/or with a low effort level, so that even if you do the work your patch is likely to be ignored (at least temporarily).
- immibis 2y ago
  You're viewing this from the perspective of a dependency resolution engine. From the perspective of a software engineer, the solution is to find better-behaved dependencies.
ForHackernews 2y ago
I think it could be very beneficial for enabling backwards compatibility: import version1 import version2 try: version2.load_data(input) except ValidationError: version1.load_data(input) I'm sure people will abuse it, but the idea doesn't seem terrible on the face of it to me.
- Drakim 2y ago
  This assumes that you will get an actual error throw and not merely incorrect behavior.
- CraigJPerry 2y ago
  Could you not do that today? import version2 try: version2.load_data(input) except ValidationError: import version1 version1.load_data(input) (Assuming version in name like this example version2 or lib2 etc.)
  - ForHackernews 2y ago
    Only if the library is renamed for each new version, which seems a bit impractical.
    - CraigJPerry 2y ago
      I guess that’s my central thesis against the need for this work - it’s not impractical to rename an import. With a tool like rope, I feel fairly confident I can refactor a source available, pure python dependency, pretty quickly. Where I get less comfortable with my idea is that not every dependency has source available (e.g. db2 database driver as an example). Another case is where some deps which have source available but the python module dependency is in C/C++/Rust - e.g. scipy
  - zahlman 2y ago
    The pseudocode in GP fails to capture the idea. Right now, two different versions of the same third-party library, would ordinarily have the same `import` name in the code. Even if you somehow hacked Pip or otherwise managed to install multiple versions of the library side-by-side in the same Python environment, there would be no way, at the level of Python syntax, to specify which one to use. The default import machinery would simply choose a) whatever's cached in `sys.modules`; b) failing that, whatever is found first by the `sys.path` search process. There are many hooks provided to customize this process, but there would be no way to specify the version to use in the `import` syntax, aside from using separate names. Of course, you can change the import name between versions. That's one of the upsides of not tying the import name to the distribution name, and many real-world projects actually do this as part of a deprecation cycle (for example, `imageio` has been doing it with recent versions, offering "v2" and "v3" APIs). But in the general case, you'd have to change it with every version (since your transitive dependencies might want different minor/patch versions for some obscure reason - semver is only an ideal, after all), which in turn means your users would always have to pin their dependency to the exact version described by the code.

baq 2y ago

You’ve never found yourself in a dependency resolution situation where there are no solutions to requirements. You need multiple versions of the same package in such cases. The alternative is just cheating: ignore requirements, install packages and hope for the best. Alas hope is not a process.

dikei 2y ago
The usual way is to put conflicting versions in optional-dependencies, and then build one target for each conflicting set of deps. That'd work fine if the code path of one target doesn't touch the others, which is often the case. You'd obviously need to have tests for both targets, possibly using a flexible test runner like `nox` to setup separate test env for each target.
theelous3 2y ago
I've never found myself in this situation where it can't be solved by being throughtful and taking the time to improve my code.
- leetrout 2y ago
  These issues crop up in dependencies more than your code. Then you have to vendor one of the deps and edit it (and hopefully ship that back upstream and hope the maintainer will merge it).
zokier 2y ago
No, the alternative is to just fix the code so that the conflict is removed. This is why open source is so powerful, you are empowered to fix stuff instead of piling up tower of workarounds.
- HelloNurse 2y ago
  You can fix your code, but indirect dependencies (you use A and B, both depend on C, but different versions of C) cannot be handled well. In C, C++, maybe Java, you would at least be able to link A and B with their own private copies of C to avoid conflicts reliably with standard mechanisms rather than unreliably with clever magical tools.
  - Too 2y ago
    You missed the point. If A, B and C are all open source you can actually fix them too. Just send a PR. Most projects are open to ensuring compatibility moves forward.
    - HelloNurse 2y ago
      Just send a PR? What are you talking about? A project that requires an old version of a dependency either has technically valid reasons, and is unlikely to be upgraded just because one more user asks nicely, or is maintained at a slow pace and/or with a low effort level, so that even if you do the work your patch is likely to be ignored (at least temporarily).
immibis 2y ago
You're viewing this from the perspective of a dependency resolution engine. From the perspective of a software engineer, the solution is to find better-behaved dependencies.

ForHackernews 2y ago

I think it could be very beneficial for enabling backwards compatibility: import version1 import version2 try: version2.load_data(input) except ValidationError: version1.load_data(input) I'm sure people will abuse it, but the idea doesn't seem terrible on the face of it to me.

Drakim 2y ago
This assumes that you will get an actual error throw and not merely incorrect behavior.
CraigJPerry 2y ago
Could you not do that today? import version2 try: version2.load_data(input) except ValidationError: import version1 version1.load_data(input) (Assuming version in name like this example version2 or lib2 etc.)
- ForHackernews 2y ago
  Only if the library is renamed for each new version, which seems a bit impractical.
  - CraigJPerry 2y ago
    I guess that’s my central thesis against the need for this work - it’s not impractical to rename an import. With a tool like rope, I feel fairly confident I can refactor a source available, pure python dependency, pretty quickly. Where I get less comfortable with my idea is that not every dependency has source available (e.g. db2 database driver as an example). Another case is where some deps which have source available but the python module dependency is in C/C++/Rust - e.g. scipy
- zahlman 2y ago
  The pseudocode in GP fails to capture the idea. Right now, two different versions of the same third-party library, would ordinarily have the same `import` name in the code. Even if you somehow hacked Pip or otherwise managed to install multiple versions of the library side-by-side in the same Python environment, there would be no way, at the level of Python syntax, to specify which one to use. The default import machinery would simply choose a) whatever's cached in `sys.modules`; b) failing that, whatever is found first by the `sys.path` search process. There are many hooks provided to customize this process, but there would be no way to specify the version to use in the `import` syntax, aside from using separate names. Of course, you can change the import name between versions. That's one of the upsides of not tying the import name to the distribution name, and many real-world projects actually do this as part of a deprecation cycle (for example, `imageio` has been doing it with recent versions, offering "v2" and "v3" APIs). But in the general case, you'd have to change it with every version (since your transitive dependencies might want different minor/patch versions for some obscure reason - semver is only an ideal, after all), which in turn means your users would always have to pin their dependency to the exact version described by the code.