3 ms·

The only changes made to PIL in Ubuntu in any current release (Precise, Trusty, Xenial) are security fixes: https://launchpad.net/ubuntu/+source/pillow https:/

by rlpb 10y ago

The only changes made to PIL in Ubuntu in any current release (Precise, Trusty, Xenial) are security fixes: https://launchpad.net/ubuntu/+source/pillow https://launchpad.net/ubuntu/+source/pillow. > The main issue is that they're modified at all Not modifying packages at all would also be a major issue. We'd end up shipping dysfunctional packages, or packages with major bugs, or not fixing bugs during the lifetime of a release. Distributions do a ton of integration work that I think goes unnoticed by users. We routinely find issues and send them upstream - every development cycle. Users don't notice because they'll also find the bugfixes in upstream releases because we sent them there. But upstream releases don't all happen in lock-step, so we end up having to carry some patches to make everything work with everything else. In the general case, we can't win. We'll always upset someone, which is why I asked for specifics. In general, we will: (0) Avoid changing anything in a stable release, so users don't have things changed on them - except for (1). (1) Fix bugs (including security bugs) as they are reported. Because of (0), this generally means that we cherry-pick fixes, which does cause some divergence from upstream. But if we don't do this, then other users complain either about the bugs or, if we pull everything into a package in an existing stable release, about things being changed under them. (2) During development of a release we fix for integration issues, since every package in the distribution is expected to work with every other package - and (though with some exceptions) we generally only ship one version of each thing in a release. This means that we sometimes need to patch things (we prefer cherry-picks from upstream, but sometimes we don't get a response from upstream in time for our release schedule) in order to make everything work. But in the specific case of PIL, we appear to not have had to do any of these things, except for some security fixes. For urllib3, it looks like we had to make some modifications to fix security issues with regards to TLS and certificate verification in Trusty, and one bug in Xenial related to broken IPv6 square bracket handling, and that's it. I don't really see how we can do anything different here. Should we have ignored the security issues and shipped vulnerable packages? Or should we have upset many more users of the stable release by bumping behaviour under the feet of existing deployments?

Eridrus 10y ago
I'm not a Linux distro expert, I'm not even a python expert, all I noticed is that my code didn't work with the OS-managed PIL/urllib3 and installing the versions from pip in virtualenv fixed things. I don't know the reasons, just that was my experience. Maybe it was just the OS versions being old, I don't know. The maintainer approach works well for everything you maintain, it just sucks for everything else, including installing out of repo software or writing your own software. It's made particularly bad by Python's packaging system which defaults to global installs. FWIW, I lay this one more at the door of Python than I do Ubuntu, you guys are just dealing with their packaging system and I think that "Python packaging is a good now" is just ignoring the shitty parts. Frankly it's easier for me to just always run my code inside a virtualenv than it is for you guys to make some drastic changes to make my life marginally easier, when I don't even really know what the problem was. It's just that this is a non-obvious thing to do for someone who hasn't been in the python ecosystem for long.
- rlpb 10y ago
  It sounds like all you needed was a newer version. Most distributions deliberately try to ship a version that was current at the time of release and don't update it later except for bugfixes. This is for the user stability reason I gave above. This is what a distribution release is, and why many Ubuntu users prefer the LTS release over every six-monthly release. If this is the case here, then this isn't a Python-specific problem at all. Developers have a choice: use the chosen distribution release version, or if that's not new enough, then find another way. For Python, one (commonly accepted and good) answer happens to be virtualenv. Many choose, because of their particular circumstances, to go all virtualenv and ignore distribution packages altogether. Can I guess that you used an Ubuntu LTS, so you effectively opted in to packages that are older but don't change often? Some solve this by never using a distribution version at all, which is fine. But understand that this is the nature of what a distribution is, and applies to the entire ecosystem, not just the Python one.
  - Eridrus 10y ago
    It's possible I had an old version, but I was trying to do super basic things with PIL on 14.04 LTS; literally just cropping a JPEG and things were failing. Do I have this problem in Java/Node/Go/Rust? No, I don't. The reason I don't have this problem is that dependencies are not installed globally, they are defined per-project and often statically linked in, there is no such thing as a conflict between two of those projects.
    - rlpb 10y ago
      > It's possible I had an old version, but I was trying to do super basic things with PIL on 14.04 LTS; literally just cropping a JPEG and things were failing. It sounds to me that you hit some kind of straightforward bug then, rather than some systemic issue.

by rlpb 10y ago

Eridrus 10y ago

I'm not a Linux distro expert, I'm not even a python expert, all I noticed is that my code didn't work with the OS-managed PIL/urllib3 and installing the versions from pip in virtualenv fixed things. I don't know the reasons, just that was my experience. Maybe it was just the OS versions being old, I don't know. The maintainer approach works well for everything you maintain, it just sucks for everything else, including installing out of repo software or writing your own software. It's made particularly bad by Python's packaging system which defaults to global installs. FWIW, I lay this one more at the door of Python than I do Ubuntu, you guys are just dealing with their packaging system and I think that "Python packaging is a good now" is just ignoring the shitty parts. Frankly it's easier for me to just always run my code inside a virtualenv than it is for you guys to make some drastic changes to make my life marginally easier, when I don't even really know what the problem was. It's just that this is a non-obvious thing to do for someone who hasn't been in the python ecosystem for long.

rlpb 10y ago
It sounds like all you needed was a newer version. Most distributions deliberately try to ship a version that was current at the time of release and don't update it later except for bugfixes. This is for the user stability reason I gave above. This is what a distribution release is, and why many Ubuntu users prefer the LTS release over every six-monthly release. If this is the case here, then this isn't a Python-specific problem at all. Developers have a choice: use the chosen distribution release version, or if that's not new enough, then find another way. For Python, one (commonly accepted and good) answer happens to be virtualenv. Many choose, because of their particular circumstances, to go all virtualenv and ignore distribution packages altogether. Can I guess that you used an Ubuntu LTS, so you effectively opted in to packages that are older but don't change often? Some solve this by never using a distribution version at all, which is fine. But understand that this is the nature of what a distribution is, and applies to the entire ecosystem, not just the Python one.
- Eridrus 10y ago
  It's possible I had an old version, but I was trying to do super basic things with PIL on 14.04 LTS; literally just cropping a JPEG and things were failing. Do I have this problem in Java/Node/Go/Rust? No, I don't. The reason I don't have this problem is that dependencies are not installed globally, they are defined per-project and often statically linked in, there is no such thing as a conflict between two of those projects.
  - rlpb 10y ago
    > It's possible I had an old version, but I was trying to do super basic things with PIL on 14.04 LTS; literally just cropping a JPEG and things were failing. It sounds to me that you hit some kind of straightforward bug then, rather than some systemic issue.