4 ms·

> However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all orig

> However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original). FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. I'm not saying here that pip doesn't have some gnarly internal details, just that the bigger thing holding it back is the lack of maintainer resources. > For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. For pip to follow suit, it would have to completely redo its caching strategy from the ground up, because right now its cache is designed to save only download effort and not anything else about the installation process. I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy. That said, I do think there are other good examples to your point, like uv's ability to prefetch package metadata, I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver.

zahlman 1y ago
Good to see you again. > FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. Fair enough. I'm sure if someone were paying me a competitive salary to develop my projects, they'd be getting done much faster, too. > I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy. I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files. > uv's ability to prefetch package metadata, You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays. > I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver. Ugh, yeah. I know the resolution logic has been extracted as a specific package, but it's been challenging trying to figure out how to actually use that in a project that isn't pip.
- notatallshaw 1y ago
  > I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files. I'm talking about the fact there is no `uv pip download` or `uv pip wheel`. I'm sure we'll discuss when you add this feature to uv, but I personally find uv's cache already grows too big too fast, so adding more to it makes me concerned. > You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays. Yeah, one of the way uv speeds up resolving is it pre-downloads .metadata files and checks if their requirements are identical to versions it had already checked, so it can quickly rule them out. It's a clever use of a collector and an advanced resolver.
imtringued 1y ago
>FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. No. This something people tell each other, that it's a lack of resources, but in reality almost all OSS projects with long standing flaws don't have a resources problem. They have a prioritization problem, where they actively ignore and refuse to work on things that affect users every single day of usage. There are features in FreeCAD that are straight up just broken that you hit every single day of using FreeCAD. When a simple low cost fix is suggested you get immense pushback, because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance. Progress happens when you sidestep those who block progress. That's what realthunder was doing and that's what Astral is doing with uv. No more bullshit excuses.
- zahlman 1y ago
  > They have a prioritization problem This is what forks are for. Alternately, money would probably help. Otherwise it's a bit rude to impose your priorities on others. Perhaps some larger projects lose track of what they intended to prioritize. This is also a resources problem. There's nobody available to do management (and probably also not a social structure internally that would allow for it). > because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance. The developers will always be in a better position than end users to assess the long-term maintenance impact.
  - quotemstr 1y ago
    imtringued is right though. Obstinate maintainership is a real failure mode of open source (and other) projects and it's often true that patch rejections involving phrases like "maintenance burden" are often pretextual. It's a shame to have to resort to creative destruction to improve things. Forks have a low success probability.
    - notatallshaw 1y ago

zahlman 1y ago

Good to see you again. > FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. Fair enough. I'm sure if someone were paying me a competitive salary to develop my projects, they'd be getting done much faster, too. > I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy. I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files. > uv's ability to prefetch package metadata, You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays. > I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver. Ugh, yeah. I know the resolution logic has been extracted as a specific package, but it's been challenging trying to figure out how to actually use that in a project that isn't pip.

notatallshaw 1y ago
> I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files. I'm talking about the fact there is no `uv pip download` or `uv pip wheel`. I'm sure we'll discuss when you add this feature to uv, but I personally find uv's cache already grows too big too fast, so adding more to it makes me concerned. > You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays. Yeah, one of the way uv speeds up resolving is it pre-downloads .metadata files and checks if their requirements are identical to versions it had already checked, so it can quickly rule them out. It's a clever use of a collector and an advanced resolver.

imtringued 1y ago

>FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. No. This something people tell each other, that it's a lack of resources, but in reality almost all OSS projects with long standing flaws don't have a resources problem. They have a prioritization problem, where they actively ignore and refuse to work on things that affect users every single day of usage. There are features in FreeCAD that are straight up just broken that you hit every single day of using FreeCAD. When a simple low cost fix is suggested you get immense pushback, because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance. Progress happens when you sidestep those who block progress. That's what realthunder was doing and that's what Astral is doing with uv. No more bullshit excuses.

zahlman 1y ago
> They have a prioritization problem This is what forks are for. Alternately, money would probably help. Otherwise it's a bit rude to impose your priorities on others. Perhaps some larger projects lose track of what they intended to prioritize. This is also a resources problem. There's nobody available to do management (and probably also not a social structure internally that would allow for it). > because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance. The developers will always be in a better position than end users to assess the long-term maintenance impact.
- quotemstr 1y ago
  imtringued is right though. Obstinate maintainership is a real failure mode of open source (and other) projects and it's often true that patch rejections involving phrases like "maintenance burden" are often pretextual. It's a shame to have to resort to creative destruction to improve things. Forks have a low success probability.
  - notatallshaw 1y ago