7 ms·
Your package manager should be boring, extremely backward and forward compatible, and never broken. Experience has shown this not to be true for python. Sever
by colechristensen 6y ago
Your package manager should be boring, extremely backward and forward compatible, and never broken. Experience has shown this not to be true for python. Several times over the years i’ve found myself, pinning, upgrading, downgrading, or otherwise juggling versions of setuptools and pip in order to work around some bug. Historically I have had far more problems with the machinery to install python packages I have had with all of the other python packages being installed combined, and that is absurd.
- whalesalad 6y agoThat is interesting. I haven't worried about setuptools since like 2009 and haven't cared about my pip version since like 2012. I have had FAR more issues with performing Bundler/Ruby upgrades than I've ever had with Python.
- incanus77 6y agoSame here.
- colechristensen 6y agoHere is one from September: https://news.ycombinator.com/item?id=24336058 https://news.ycombinator.com/item?id=24336058
- user5994461 6y agoThat one looks like an issue with the debian package trying to alter paths and blowing things up. There's definitely recurring issues with distributions trying to handle python dependencies, which doesn't match how python handles dependencies.
- LevGoldstein 6y agoI take it you're doing a lot of greenfield projects with the latest versions of Python then. I've encountered issues with Setuptools as recently as ~3 months ago: https://github.com/pypa/setuptools/issues/2352 https://github.com/pypa/setuptools/issues/2352 I've also had to play the pip version pinning game a few times in the past.
- acdha 6y agoThis seems like an odd comment to make on a release which is making improvements, work done with the full knowledge that there’s room for improvement. It’s also not universal: it’s been a decade since I’ve had to worry at all about that and even before then it was uncommon except on projects which weren’t trying to be very clever. If you had a clean setup this really hasn’t been the rule for a long time.
- disgruntledphd2 6y agoAs long as you don't have lots of C or Cpp dependencies, Python packaging is apparently totally fine. Unfortunately I don't live in that world :(
- acdha 6y agoI mean, I do use a lot of C code and the things which break are external C level dependencies. You can blame Python for those but it won’t help fix them.
- disgruntledphd2 6y agoWell when pip installs an incompatible version of Numpy relative to my current setup (without warning me) then I do indeed blame pip. To be fair, this release makes that much better, which I am extremely grateful for.
- acdha 6y agoIf a library doesn’t specify version compatibility and you don’t pin it but do choose to run an upgrade, why is that the installer’s problem? It’s like complaining about Toyota when you get a parking ticket.
- disgruntledphd2 6y agoSee, this isn't necessary in other languages. All I want is an error, so that I can assess what the right thing to do is. Honestly, I sometimes feel that people have stockholm syndrome with python packaging. Nonetheless, I'm really happy about this release, as it will help to prevent these problems. It's also worth noting that I didn't even learn Python for a long time because I kept breaking my install while trying to get started, so this stuff has real costs in terms of adoption.
- ransom1538 6y agoHave you tried PHP? It has an amazing package manager called 'composer'. The current release is popular and super amazing (runs 1/2 the internet). Python is a terrible divided (2 or 3?) and gross language from 1991.
- ransom1538 6y agoheh. the down votes are so pefect.
- fest 6y agoI remember composer as the package manager that required a gigabyte of RAM to install a few dozen of dependencies for some framework.
- rovr138 6y agoIt can definitely be more than 1 now. New version should help, but also breaks. Source, personal experience with my projects.
- mech422 6y agoInsert "Poorly Hacked Perl" joke here ?? /s ohh - lets do vi vs emacs next! :-)
- westurner 6y ago"Experience has shown"? Did you go create a test case? Or at least link to a specific issue?
- woah 6y agoThe entire Python ecosystem is like this. Different conflicting versions of the language, different package managers broken in different ways, etc. It’s so bad that the core user base of data scientists usually just use machines that have all their libraries pre installed. Contrast this with a language like Rust or JS, where you run one command once to install everything
- ubercore 6y agoI don't know Rust well enough to comment, but doing a lot of development in both JS and Python, JS is no cakewalk either. I'd say I deal with at least as many packaging/versioning issues on JS as I do on Python.
- zo1 6y agoThe JS ecosystem has just as many "issues" if you look at it objectively. Don't mean to be funny or lazy, but one need only look at this SO answer to see how weird/fragmented the JS package system is (the answer has a Table of Contents). https://stackoverflow.com/a/39825582/1140382 https://stackoverflow.com/a/39825582/1140382
- woah 6y agoThat's not the JS package ecosystem. Those are all various techniques from 5 years ago for packaging frontend code to save download bandwidth.
- meowface 6y agoAnother issue is the two mutually incompatible ecosystems of async vs. non-async Python libraries and frameworks. You now have these entire mirror universes of code, with "[X]-async" versions of existing synchronous libraries, and new async frameworks which require using these libraries. The years of work that've gone into non-async libraries are now basically all for naught, with lots of people reinventing the wheel for every possible thing that interacts with a network. (I'm also not a fan of async/await in general.) I'm a big Python fan; it was my first language and still by far my most-used and the one I know best. But the "developer experience" is undoubtedly a giant mess compared to something like Go or Rust. (Rust also appears to be having some growing pains related to async, but everything else seems way more solid.) Despite Python's catchphrase "there should be one, and preferably only one, way to do it", Go really embodies this across every dimension. One way to install the runtime, one way to format, one way to package, one way to do concurrency, one way to test, one way to write most things. And every version is almost completely backwards compatible. It's kind of crazy that a language that (justifiably) prides itself in its programming simplicity also makes you deal with tangled messes like https://xkcd.com/1987/ https://xkcd.com/1987/ My one solace is that some all-in-one third-party tooling is finally close to the level of convenience you get with Rust. pyenv and poetry save a ton of headache when it comes to managing Python versions and packages, and "just work" in a pretty similar way as rustup and cargo.
- munificent 6y agoI don't disagree with your claim, but like all moral claims, I don't see how it's actually useful or actionable. Children should not get cancer. People should not drive without seatbelts. Our optic nerve exit the back of our eye instead of having to go through blind spot on our retina. And yet here we are. The Python ecosystem is older than our modern conception of how a package manager should behave. Any new package manager must build on top of that chaotic history. Absent a time machine, your comment just seems like a vague lament about the state of the world.
- d0mine 6y agoActionable: use backward-compatible behavior by default. Use opt-in for new features. Question: is the new resolver opt-in or is it enabled by default in the new pip version?
- colechristensen 6y agoIt is enabled by default. I think there was a beta period where it was an option.
- user5994461 6y agoAnd there is an argument to disable the check.
- xapata 6y agoHow long should someone wait before deprecating the old behavior?
- colechristensen 6y agoThe action is simple, stop making breaking changes. There is frustration with breaking changes being in the history, but that is much less troubling than the reoccuring answer to the question “What is wrong with my code?” being “A new version of the package manager was released.” It is also very possible to write code to completely work around old issues so the newest version of your package manager just works or at least outputs clear error messages with mitigation steps, but instead what you get is obscure stack traces when you run pip install.
- suyash 6y agoinstalling code, packages is the hardest part of python programming, one of the reason I avoid it.
- zo1 6y agoIn my experience, python dependency management + library installations + code sharing is way easier than the following: C#, C++ (Conan) and Java, Linux(apt, yum, etc) and most certainly JS/TS. What is so hard about "installing code, packages" in python for you?
- suyash 6y agoJava - all you have to do is mvn install clean build -> Maven is amazing at dependency management. Same in JS - npm install and npm start works like a charm.
- zo1 6y agoAnd python all you have to do: pip install -r requirements.txt Or even better: pip install -e /path/to/my/project Works like a charm.
- suyash 6y agowhat about version management when conflict happens in dependencies that are installed vs needed. Admit Python is a big mess. That is why people recommend installing within another weird software called conda environment vs directly.
- zo1 6y agoCorner case, and a tad overblown from an "academic" perspective if you ask me - I've been doing day job (and after-hours hobby with ML/CV) development in python for the last 15 years and had that issue maybe 2-3 times. On the other hand, with my very limited exposure to my team's JS/TS FE components, I've almost each time encountered issues with the supposedly easy "npm install" command. Broken NPM installs, cryptic error messages (ok get this one on Python too tbf), lingering version lock files, global vs local package issues, broken nodejs installs, node_modules being committed, node_modules being un-deletable, etc. Anecdotally, I'd say my experience on Python packaging and management has been worlds-better than for other languages. Python has it's issues sure, but from my point of view, packaging is not a big one in comparison to other langs.
- Der_Einzige 6y agoMaybe I am weird, but I've had far more problems with anaconda and its package management than I ever have with pip. Conda routinely takes hours to resolve whatever unholy issues it's running into sometimes
- jnwatson 6y agoIn other words, your package manager should be perfect and bug free 5 years ago. There are zero package managers from any environment for which these requirements are met.
- riffraff 6y agobut python has been around for a while, I feel the frustration comes from the fact that other platforms seem to have managed to get a single working tool much earlier.
- jnwatson 6y agoThe only platforms that have excellent package management systems are the ones you haven’t used yet. Npm is probably the closest. They had $10.6 million of investment to get it right.
- stingraycharles 6y agoI’m fairly certain that yum and apt meet these requirements for “traditional” package managers, I know Maven does as well. I think it depends a lot upon the culture of the ecosystem, maven being fairly conservative is a natural consequence of Java being very enterprise focused.
- viraptor 6y agoThat's easy to disprove. See the bugtrackers for those projects. They're not bug free. See why "aptitude" was introduced to cover "apt"s failures. Also one of the reasons we had the yum to dnf migration is to replace the worse resolver in yum with a sat-solver in dnf.
- bogwog 6y agoIt's mind boggling to me that such a popular, mature, and beloved language has such a terrible package manager. Pip is seemingly broken more times than not. Deploying anything is a nightmare unless you use a virtual environment, and even when you do you still gotta cross your fingers and hope pip doesn't decide to break at some point in the future.