25 ms·

“Python's batteries are leaking”

• axaxs 7y ago
  Why, time and time again, does Guido seem incapable of reasonable debate, or ideas that challenge his own? It's completely rude to interrupt a presenter with 'what is your point?' Years ago I was in contact with the author of Nuitka, who was very excited to share his work thus far. During his presentation, Guido kept huffing and making snide comments under his breath. All because he disagrees with the premise behind Nuitka. I like Python, and can appreciate his work and contributions. That said, I can't help think the community can become less toxic without him as BDFL.
  • michael_j_ward 7y ago
    Guido resigned as BDFL in July of 2018[0]. [0] https://mail.python.org/pipermail/python-committers/2018-July/005664.html https://mail.python.org/pipermail/python-committers/2018-Jul...
    • axaxs 7y ago
      Apologies if it wasn't clear, but that's what I was alluding to. A lot of folks, and to an extent myself, feel that it will become a worse, design by committee language. I was just trying to point out the upside.
  • tyingq 7y ago
    Reading the article, she seemed to handle it well, and he stormed off.
  • pavlov 7y ago
    Maybe he’s simply burnt out? The pressure of being in charge of something as big as Python must be intense. People make demands on his time and expect the “benevolent dictator” to cunningly solve everything like a modern day Salomon. I know he gave up the title, but as he personifies Python, I imagine the influx of requests for his attention may not have subsided much.
    • mlevental 7y ago
      how is that an excuse for being uncivil? if you're burnt out then don't attend the talk? simple.
      • pavlov 7y ago
        From my own past experience with depression, it’s not as simple as “I’ll just stop doing all the things that have kept me in motion until now.” This is not meant to speculate on van Rossum’s health, just trying to point out that there are situations where apparent rudeness may be out of the person’s immediate control.
        • newen 7y ago
          You can say the same thing about anyone doing anything. It's just giving excuses for bad behavior.
          • yjftsjthsd-h 7y ago
            Yes. It is an excuse, even for poor behavior.
        • geofft 7y ago
          If you're sufficiently in control of your actions to keep developing a major programming language, you're sufficiently in control of your actions to either not be rude while doing so or be aware that you're going to be rude and need to find a way of dealing with the situation (avoiding it, taking extra time before speaking, working with a therapist, whatever). Mental health issues don't excuse bad behavior. (Also, personally, if I were behaving poorly I would much prefer people to say "hm, 'geofft is being rude but I know he's a better person than that and can improve" and not "hm, 'geofft is being rude, that's just the way he is, I wonder if he's got mental health problems.")
          • phyzome 7y ago
            > Mental health issues don't excuse bad behavior. Basically agreed. > If you're sufficiently in control of your actions to keep developing a major programming language, you're sufficiently in control of your actions to [...] not be rude while doing so [...] That's a lovely thought, but no. When I've had bad times, my social skills and emotional cope would sometimes go to shit and leave my technical skills intact.
  • userbinator 7y ago
    It seems that those programming language communities which have a more "social" aspect also tends to lead to more associated drama like this.
    • FranzFerdiNaN 7y ago
      It's one of the things I really like about the R community. It's incredibly friendly and without egos, and has lots of people who actively work to make the community as safe and welcoming for everybody. I can't say I've ever heard about drama like this from there.
      • new4thaccount 7y ago
        R is one of the few languages that I actually use a little bit, yet know absolutely nothing of the community and history. I mean I could tell you all about a dozen niche languages from Lisp to Haskell, but nothing of this masterpiece. Time to go read up on R :). Any chance you could give a short little blurb for how the community is organized and works? I'm curious how different it is to Python. I also sometimes worry about the future of R where you have Python-Pandas coupled with Spyder IDE and Julia-DataFrame library with Atom-Juno as IDE as well as JuliaDB. It seems like a lot of communities are moving in on what R does best.
    • geofft 7y ago
      Seems like there might be some other confounding factors here: - A language that is simply less active or whose users feel less ability to have a say in the language's development is going to be both less social and less dramatic, but also carries a high chance that existing users are only there for legacy reasons and are always considering moving to another language better suited for their purposes. - A language run by a corporation is less "social," and has the drama play out in business negotiations, closed-door committee meetings, and lawsuits instead of on blogs and public mailing lists and (in this case) closed-door meetings with an expectation of public reporting. I'm not aware of any drama in the ASP community, but that probably also had something to do with many users having picked Python for their next project instead of ASP. I'm aware of some drama in the Clojure community, the conclusion of which seems to have been that Clojure is their corporate sponsor's language and if it doesn't work for you you should find something else.
  • stuaxo 7y ago
    Yeah, it's disappointing, he seems like a bit of a dick TBH. (Though in nothing on the level of some in open source).
  • mixmastamyk 7y ago
    It may not make complete sense, unless you are Dutch.
  • mixmastamyk 7y ago
    What’s your point? Because it is easy to criticize an important work. Not so easy to help improve it or leave it be due to historical and compatibility concerns.
• deleted 7y ago
  [deleted]
• isuckatcoding 7y ago
  Wow this is not the conduct I expect from a language creator. I don’t care if you’re Albert Einstein. Humility and being able to take criticism is far more admirable to me.
  • hjk05 7y ago
    He asked her twice to be more specific about what she was arguing, which seems fair after a long tirade of shitting on anything and everything non optimal about the stdlib without any specific point. And he then left during Q&A which is totally fair, he’s his own person and it wasn’t his Q&A, people are blowing this out of proportion.
  • mceachen 7y ago
    We weren't there. He might've needed to be someplace else. People leave all the time during talks, and I've been to talks where most people left as soon as Q&A started.
  • mychael 7y ago
    What "conduct"? It's a simple disagreement, not some big dramatic fight.
  • xtreak29 7y ago
    I think he was probably frustrated by Python 2 support. If twisted has 50% userbase in Python 2 that don't upgrade and requiring more volunteer time on Python 2 just handcuffs language development on Python 3. Python 2 has been deprecated for long time and if people still want free Python 2 support I think it's just indigenous to the language contributors to spend their already constraint free time on Python 2.
• resoluteteeth 7y ago
  When I first used python like 20 years ago I was blown away by how much functionality was blown in, and it can be annoying using languages where even the most basic functionality involves downloading 50 packages from the internet, but on the other hand the standard library does seem to be a mess now.
  • new4thaccount 7y ago
    Yea. I'm using Rust at the moment for fun and the amount of things not in the standard library is crazy to me. What? There is no built-in dictionary? What do I use instead and where is it? Edit: based off of all the replies below, everyone understands the validity of what I'm trying to say, but also have fortunately pointed out my admittedly grevious error of not knowing you can just import hashmap from stdlib. The extra step is pretty minimal and not a problem. I'm hoping this is covered in the Rust book.
    • resoluteteeth 7y ago
      Yeah Rust is pretty extreme. I was just trying to generate a random number in it and was pretty surprised to find out that at some point it did have this functionality in the standard library but they actually removed it and now it's a separate crate.
      • nicoburns 7y ago
        That is actually being considered for inclusion at some point. It was mainly excluded due it not being ready and them not wanting to commit to the existing API. That said, I like the Rust way. Including a library is pretty easy, and it mains you get the best API as the main one (rather than a stdlib function that is "good enough", and another library to use if you really care about that functionality).
    • jzoch 7y ago
      You mean a map? It's in the standard library.
• jteppinette 7y ago
  > six is non-optional for writing code for Python 2 and 3 I maintain a Python 2 & 3 compatible project that has no external dependencies.
  • eikenberry 7y ago
    Same here, though mine is a small project. I'd guess six probably becomes more useful when you starting hitting certain features/areas of the language. I'd be curious about what things make six start being useful.
  • dpkp 7y ago
    have you written your own 2/3 compatibility layer? I can't imagine writing anything large without six... fwiw, six can easily be vendored into a project to avoid the technical external dependency. that is how we manage it for kafka-python.
    • jteppinette 7y ago
      No compatibility layer. Its really not bad. there are a few modules that are simply at different locations but have the same API if PY3: from http import client as httplib else: import httplib constants PY3 = sys.version_info >= (3, 0) PY2 = sys.version_info < (3, 0) PY26 = sys.version_info >= (2, 6) and sys.version_info < (2, 7) is string isinstance(<maybe_string>, basestring if PY2 else str) using different classes # Python 2.6 doesn't properly UTF-8 encode syslog messages, so it needs # to be performed in a custom formatter. formatter_class = UnicodeLoggingFormatter if PY26 else logging.Formatter
    • X-Istence 7y ago
      You don't need all of six in most cases. You just need a couple of different functions. As the maintainer of WebOb/Pyramid/Waitress we have a compat module that contains all of the changes/renames/functions to help with the Python2/3 compatibility and all tests run across both platforms. Are the functions borrowed heavily from six? Yes, but we don't need to vendor all of six.
  • mehrdadn 7y ago
• nurettin 7y ago
  It took me an hour to create a program that tails some logs and alerts when it doesn't receive any logs for a given amount of time. For this task I did not even need to leave the asyncio module. It lets you create subprocesses and execute call_later on the event loop in order to simulate a heartbeat while reading the output of tail at the same time. Did asyncio module feel bloated? It certainly did. It seems like every module from subprocess to networking to io is crammed into it. On the other hand, did it get the job done without resorting to any packages or threading? Yep, and that is pretty powerful and rare.
  • Waterluvian 7y ago
    Asyncio is an amazing tool that makes me not hate doing async with Python. There was quite a learning curve with the library. A lot of Lego pieces. But I found the handful I need and then learn new ones occasionally. I think bloatedness of the stdlib isn't actually a practical problem. It's just an inelegance that you kind of have to learn to tolerate.
• kentm 7y ago
  I was rather shocked to find that python didn’t have a full-featured crypto library included in its standard lib. The alternatives all ended up being unmaintained or maintained by small groups (which makes trust in the soundness difficult). I tapped our security team, who were in disbelief, but ultimately they gave up to and I wrote the software in go instead.
  • tptacek 7y ago
    You want pyca/cryptography. The last thing in the world you want is a standard crypto library that no experts are enthusiastic about maintaining. Golang had an unfair advantage here, because the language team included cryptography engineers. It would be weird if most languages had the same kind of crypto in their stdlibs. I think there is in general nothing wrong with a language ecosystem where key parts of the whole platform are in well-maintained third-party libraries rather than the standard library. Which is also something Amber Brown is saying here.
    • jteppinette 7y ago
      When you find a coworker trying to use this module.. > cryptography/src/cryptography/hazmat/__init__.py Hazardous Materials This is a "Hazardous Materials" module. You should ONLY use it if you're 100% absolutely sure that you know what you're doing because this module is full of land mines, dragons, and dinosaurs with laser guns.
    • geofft 7y ago
      Also I expect that a) pyca/cryptography is maintained by as many people as Go's cryptography libraries are, so the worry about maintenance by small groups doesn't argue in favor of Go here b) the number of people maintaining any particular module in the Python standard library is very small (e.g., IIRC there's one maintainer for `ssl`). I would be surprised if either 'kentm or their security team wanted the rest of the Python standard library developers (or the Go ones, for that matter) to mess with cryptography modules when it's not their area of expertise.
      • 7y ago
• jteppinette 7y ago
  IMO, Golang does the best job of maintaining a high quality standard library. I disagree that modules should be moved into the external package ecosystem. However, Go isn't preinstalled on most systems like Python. I have to develop enterprise software that runs across a wide range of platforms, and being able to take advantage of the fact that Python is pre-installed on all of these systems with its standard library is a godsend.
  • nerdwaller 7y ago
    It’ll be interesting to see how Go (Rust, and other new languages) evolve and if they can avoid some level of package decay when they reach the age of Python, Java, etc.
    • frou_dh 7y ago
      Notably, a number of packages in the Go standard library have officially become "frozen": https://www.google.com/search?q=site:golang.org/pkg/+"frozen" https://www.google.com/search?q=site:golang.org/pkg/+"frozen...
    • pixelrevision 7y ago
      I’ve been working with go a lot lately and they seem really focused on not letting this happen. Every single thing in the language and standard library seem completely focused on minimalism and compiler time. The standard lib is unlikely to change all that much and people are not picking the language for a bunch of convenience features. Third party package problems will be an issue at some point but that’s more due to them be so focused on minimalism they don’t have clear guidance on setting up and maintaining packages.
      • teek 7y ago
        3rd party packages are already a problem because a github repo shouldn't be treated as a dependency source. Gomod solves some problems but still uses git repos as the source. The primary reason Go can get away with this strategy is because the Go community actively promotes fewer dependencies = better. So if you write Go you have to often accept the fact that the second you add a 3rd party dependency that you're now officially on your own if that dependency breaks or becomes unsupported. This is not necessarily a bad thing. But in order to move software forward I still think we can do better than to push this responsibility to all individual end users. This is one area where I feel like most popular languages today still fail compared to CPAN. CPAN's value was not just packaging and distribution, it was an integrated test report pipeline and infrastructure, actively managing and gatekeeping of library maintainers, CPAN mirroring functionality, and easy acceptance of bug reports and user feedback against a library.
• jMyles 7y ago
  Guido is a good dude, through-and-through, despite his perhaps bad behavior here. Amber is nothing short of an open source hero, having brought Twisted, one of the best open source projects in the world, to new heights. Her insights are as important as anyone in the python community, and after six consecutive PyCons sprinting at the Twisted table (including literally in a chair with Amber to my left and Glyph to my right earlier this month), I consider Amber's voice to be one of the truest and clearest among the leadership of the language into the future. Amber and Guido are both beautiful human beings. In the dispute that is the topic of this blog post, Amber is basically totally right. Moreover, the distinction has less to do with any kind of nagging python 2 holdover than this article suggests. The standard lib's role as a place where code goes to die is a view that is widely held and accurate for many cases. The following question went unanswered during the Steering Council Q&A: "Every feature request has a constituency of people who want it. Is there a constituency for conservatism and minimalism?" ...and that's really what this whole thing is about.
  • alexdong 7y ago
    The problem with rants is it stings and it divides. When it comes to constructive criticism, I think Amber did a good job with her criticism but can do better at the constructive front. Her problem statement was spot on and I agree that the direction she proposed is a good one. However, to separate the standard library from the core is probably even more dramatic than the Python 2 to 3 migration. Is that what the community can afford at the moment? What's needed to make the transition? What's the opportunity costs? i.e. what other developments we can do for a bigger impact? What are the pros and cons?
    • tptacek 7y ago
      Is "embrace PyPI and move things like asyncio there" not a constructive suggestion, or is she sort of being penalized because the most reasonable solution to the problem can be described in less than half a sentence so it's seems like there's more complaint than solution?
      • 7y ago
• nerdwaller 7y ago
  > She thinks that some bugs in the standard library will never be fixed. This is actually an interesting paradox to be in, and one that Linus Torvalds recently commented on. His focus, like Guido’s, is the user and even fixing a bug can break the user. https://lkml.org/lkml/2018/8/3/621 https://lkml.org/lkml/2018/8/3/621
  • notatoad 7y ago
    This isn't a paradox. once it's released it's not a bug anymore, it's just behaviour. document the behaviour, but breaking compatibility with previous versions is a bug. it doesn't matter how obviously wrong the previous behaviour is.
    • nnq 7y ago
      > but breaking compatibility with previous versions is a bug That's how you get an inconsistent mess that never evolves. There's something called semver, increase the version number and do the fix / refactors / radical redesign / whatever. People will see that you've went from version 1.0 to 87.3 in one year and they may choose not to use your thing because you're moving too fast for them, but that's life...
      • cameronbrown 7y ago
        Better than breaking software. Linux is far more important when it comes to ABI stability here than Python though.
        • adontz 7y ago
          Linux has no stable ABI :-)
          • cameronbrown 7y ago
            For driver developers sure, but it's userspace ABI is very stable.
• yingw787 7y ago
  I agree with Amber’s point that more stuff should be moved from the standard library to PyPI. I made my first pull request to CPython during the development sprints this year, and it’s honestly not the best experience. Everything is built from scratch in CI after every commit, even a documentation change. There’s nowhere near enough CI builds and pipelines for everything Python supports. Pull requests are outstanding for several months, and there’s at least a thousand PRs open when I checked this morning. I’m not sure if Python’s ideal solution is to reduce stdlib and have endorsed packages in PyPI, but it would be an improvement over the current process.
• epx 7y ago
  Why is it so difficult to admit Node.js did the package thing right, by keeping a local folder just for the app, isolation from other apps with zero effort?
  • bdcravens 7y ago
    It's worth noting that npm was developed independently of node https://groups.google.com/forum/?hl=en#!topic/nodejs/erDWyS4xPw8 https://groups.google.com/forum/?hl=en#!topic/nodejs/erDWyS4...
  • zbentley 7y ago
    I think people conflate NPM-as-package-installation-system, which I would agree works very, very well (though a lot of that is the JavaScript module/import system in general and not NPM specifically), with two other things: NPM the web platform (which has had a lot of pretty severe security/community issues), and the JavaScript community's tendency to proliferate lots and lots of modules, many in competition, to solve problems that other languages' communities tend to solve either via reimplementation or via the standard library. I think any discussion about NPM or JS packaging compared to other package managers needs to discuss those things as orthogonal, largely unrelated concepts. Otherwise everyone just picks a favorite punching bag (e.g. left-pad) and talks past each other.
  • hjk05 7y ago
    You comment has nothing to do with the article. Also I believe there’s a pep working on that, but again that has nothing to do with what does and does not go in the standard libs.
  • twic 7y ago
    The right thing is a machine-global package cache that can hold multiple versions of each package, and loader machinery that can pick out the right ones. That means you only have to download and store each package once, and you never get interference between projects. NPM installs (and downloads?) a copy of each package for each project. This is the wrong thing to do.
  • thrower123 7y ago
• peterwwillis 7y ago
  It's funny to me that they're making a point that PyPI is better than core, because actually I think PyPI has created a rather crap ecosystem. The non-hierarchial organization of packages, the lack of curation, lack of inheriting past functionality and extending it as more standard functionality, etc has resulted in a confusing sprawl of packages with duplicate, incompatible, buggy functionality. It's a bit like Linux internals; it's grown haggard over time, isn't organized well, is badly documented, and so it's difficult to pick it up and use it without stumbling over a decade or more of stale documentation and obsolete software. Perl has a much better set of modules that extend standard functionality, which considering how much flack Perl gets for being hard to read, is rather funny. Rather than every new feature being its own independent project, most of the useful modules inherit a parent and follow the same convention, leading to very simple and easy to use extensions. And Perl Core isn't all that great, but it does have some batteries included, and everything else is extended easily and in a more standard manner by CPAN.
  • zbentley 7y ago
    Wow, I've had a really opposite experience with CPAN modules. I've overwhelmingly found them to not respect encapsulation (messing with all sorts of global state, not mentioning that they're doing it, and failing to clean up after themselves or even provide the tools to clean up well), be massively inconsistent in their APIs, have messy and hard-to-parse documentation (still better than Python's conventions here, though), and have some really silly hierarchy-related decisions, most of which I suspect stem from inter-maintainer politics and infighting, of which I've observed a large amount. Sure, I've found some gems on CPAN, but, having worked on both Perl, Python, and Java at reasonable scale for awhile, I cannot understand all the praise CPAN gets. It's the worst-quality scripting language package ecosystem out there. Even NPM does a better job, and some things about NPM are awful. CPAN might have been the first/only/best package manager for a get-shit-done scripting language at some point, but not any more. Separately, I agree about modules which extend language functionality (e.g. class systems, async programming, runtime typing) specifically. Perl does pretty well in that area. While many of those language-extension modules really don't play well with any other metaprogramming tools being installed in the project, I don't imagine that any alternatives in other languages do, either. My main beef above is with "simple" (read: not pervasive semantics changes) modules like IPC utilities, HTTP clients, or loggers that don't know how to stay in their lanes.
• avar 7y ago
  She seems to be advocating that Python do pretty much what Perl has ended up doing, which is "we have some batteries, but we haven't been adding new ones for a decade or more". The reasons are similar, it's a constant drag on core compiler development to need to support various batteries included that most core contributors aren't going to care about, so it's easier to tell people "use CPAN". There was even talk of "distros" for the interpreter. Where the core bits would be similar to what Linux is, and all the batteries would be provide as collections of add-on packages. Strangely enough these efforts seem to stop at OS distributors. They really seem to like to install just the one "compiler", and wouldn't stand for a project like Perl or Python telling them "we mean for you to distribute the core compiler plus these 100 packages, because that's what forms our 'language'". "Strangely" because you'd think they'd be the best positioned to make easy work of packaging up such a thing, and it shouldn't in principle make a difference if you need to install 100 RPMs / APTs by default.
  • jzl 7y ago
    The idea of "distros" for python is interesting, and to a certain extent has already happened: just look at Anaconda. I've been using built-in environment isolation tools such as virtualenv for ages but have recently switched over to using miniconda for all things python. Among other things it has amazing support across all three major OS's, and I happen to be dealing with all three at any given time. Whether one uses miniconda, pipenv, virtualenv, or anything else like it, as far as I am concerned the days of ever using the system python are over. I will always create my own personal "distro" on the fly with full control over the python version and every add-on package.
    • imiric 7y ago
      As a non-scientific user of pyenv[0], would I benefit from switching to Anaconda/miniconda? [0]: https://github.com/pyenv/pyenv https://github.com/pyenv/pyenv
      • maximente 7y ago
        not likely IMO. i've found conda - which is their environment management tool - to be a hassle unless one needs specific numpy/scipy/GPU libs. i'm using pipsi and pew, although i'll look into pyenv.
• mjw1007 7y ago
  If your project has any third-party dependencies, and so (nowadays) you're going to set up requirements.txt and virtualenv and whatever anyway, I can see that you're going to think things like "this XML parser in the standard library is just getting in the way; I can get a better one from PyPi". But I think a lot of the value of a large standard library is that it makes it possible to write more programs without needing that first third-party dependency. This is particularly good if you're using Python as a piece of glue inside something that isn't principally a Python project. It's easy to imagine a Python script doing a little bit of code generation in the build system of some larger project that wants to parse an XML file.
  • nullwasamistake 7y ago
    What they need is an Apache Commons or Guava of Python. They're both defacto part of the standard java library.
    • dehrmann 7y ago
      I try to avoid Guava because they have a habit of making incompatible breaking changes, and because so many libraries depend on it, it's likely to cause version conflicts. The way Apache Commons puts the major version in the package is much better in that regard.
      • nullwasamistake 7y ago
        I have not experienced this running guava 16-23 in various apps. Maybe incompatible but they're good about security patches for old versions. I have never seen a version conflict between guava releases
        • BeeOnRope 7y ago
          It's very easy to get a Guava version conflict because (a) Guava frequently adds new stuff, and (b) Guava semi-frequently deprecates and removes stuff a couple versions later. So all you need is one dep that needs Guava version X with method M that is removed in version X+2 (say) and another dep that needs something new introduced in version X+2, and you have a Guava version conflict. That's, Guava releases are not backwards compatible due to removal of classes and methods. You can sometimes fix this with a technology like shade or OSGi or whatever to allow private copies but it does not always work.
• _hardwaregeek 7y ago
  I've wondered about standard libraries for a while now. What happens if you discover a security vulnerability in your stdlib? Presumably you'd have to bump the language version, deploy it out and beg users to upgrade. Except, users don't upgrade stuff. While if you version the standard library, every new project will get the newer version of the standard library. Sure, there's space tradeoffs, though you could offer a manual linking option. But at the very least, the amount of new projects with the vulnerability will be next to nil. And what if the standard library just gets dated? Take Node for instance. The fs module has a whole bunch of outdated callback based functions. Sure, you can wrap them in promisify, but it sucks that we have these outdated functions stuck around forever. There's definitely tradeoffs with package/dependency multiplication, but I don't think standard libraries are as clear cut as people make them out to be.
  • perlgeek 7y ago
    > I've wondered about standard libraries for a while now. What happens if you discover a security vulnerability in your stdlib? That depends on what you are writing. For an application that is deployed stand-alone, you'll likely fat-package it with python and all the libraries. In case of a security issue, you create a new version of your application that bundles the fixed python. For an application that is deployed on the system python (more typical on Linux), it's the system admin's task to update the system python.
    • _hardwaregeek 7y ago
      I'm talking from a language maintainer perspective. From a user perspective, sure, you can upgrade. But that doesn't mean everybody will upgrade. In general, people don't like upgrading. That's why browsers all automatically update these days.
  • will4274 7y ago
    You're trading things to update. Either the bug is in the standard library and you need to update the standard library or the bug is not in the standard library and you need to update not-the-standard-library. Either way, the maintainer of the library can't get everybody to upgrade.
• KaiserPro 7y ago
  I clicked on this link think "uh oh standard ill informed rant post" However brown has solid good points. The brilliant selling point of python is the massive standard lib. If the quality of the libraries fall, then python's use as a tool drops dramatically. One of Node's massive failures is that is has no standard lib.
  • mceachen 7y ago
    Coupled with weak infrastructure around third-party library selection. npms.io has a "quality" score (which npm pulled in recently), but that magick number includes things like download counts, if the homepage is on a custom domain (!?) and if the readme has badges (!!?). It doesn't include code quality metrics, if the package or git repo is using GPG signing, or clear signs of abandonment, like N ignored PRs, or N,000 ignored open issues. I've been astounded how few non-trivial packages are actually in a consumable state, and how many seemingly-simple packages have N dependencies that pull in M more. By and large it's a zombie wasteland of cruft. I wouldn't really trust only-crowdsourced ratings, but I think that might be a nice component for npms.io to include, perhaps. Stackoverflow answers, for example, seem to be directionally correct if you sort by upvote count.
• hashhar 7y ago
  I think the best model i have seen for a lean stdlib has been that of Golang. You have the standard lib and then you have the packages under Golang.org/x/ which are experimental packages that sometimes end up being merged into the core language. The stuff which is not in the stdlib (TOML, yaml etc.) have been supported very well by community packages.
• tanilama 7y ago
  > Brown called out the XML parser and tkinter in particular for making the standard library larger and harder to build, burdening all programmers for the sake of a few Tkinter needs to go...There is very little reason except for the legacy ones, why it needs to be there still...
  • smitty1e 7y ago
    There is the expectation of IDLE, no? Isn't that a tkinter product?
    • sigzero 7y ago
      It is.
    • tanilama 7y ago
      But this only can't justify that tkinter being part of the STANDARD library. Taking tkinter out of the standard library won't stop IDLE being built with it, if they chose to do so.
      • detaro 7y ago
        Depends, do you consider IDLE part of the standard library? (it's separated in many distributions, but documented in the standard library documentation, so it's a bit mixed, but it's probably okay to not consider it part of the library, but just the standard distribution)
      • dragonwriter 7y ago
        Though OS packaging may frequently obscure this because OS packagers break up the language distribution, Idle is part of the language distribution, so (unless you expand the distribution to include core packages outside of stdlib, as with Ruby’s gemification effort) it has to be built with stdlib alone. Tkinter, therefore, needs to be part of stdlib.
        • tanilama 7y ago
• pariahHN 7y ago
  I may be terribly wrong about this, but I would think that in general if someone makes an improvement to something you make then you would want to integrate that person and their improvement in some way. Treat like any other update: mention it in the version notes and warn about compatibility of code using the previous version. I know that renovation can suck but it's something that we need to be doing. A comprehensive stdlib means that once you've got it, you don't need to worry about being able to access packages along the way - how are you going to download a package if you can't connect to the repository? How much can you trust a third-party dev vs the core team? If a package is really niche, it may not make sense to put in the integration work. But for a package that is used by a significant majority in a general application - why would you want to keep it separate if it is so much better? I am ignoring human interaction here - there are probably of dozens of answers to that question if you count personal motivations.
  • latortuga 7y ago
    > How much can you trust a third-party dev vs the core team? Seems like a false dilemma to me. The core team could still maintain "blessed" packages that don't ship with the default installation. > why would you want to keep it separate if it is so much better? This is addressed in the article, most of the 5th paragraph is dedicated to it.
• dragonwriter 7y ago
  Note that similar issues were raised with Ruby stdlib, which is being addressed in part with “Gemification” of stdlib, so that all of stdlib (targeted for 3.0, though it's been going on since 2.4)[0] is being moved out to externally-updatable packages that are included by default (default and bundled gems), so that it is still “batteries included” but the batteries are at least replaceable. Amber's suggestion seems to be in the same direction (though perhaps not as extreme.) [0] https://www.slideshare.net/mobile/hsbt/gemification-for-ruby-2530 https://www.slideshare.net/mobile/hsbt/gemification-for-ruby...
  • bsder 7y ago
    Python cannot be atomized effectively, and the issue is political. The problem is that I cannot count on being able to install new software in many environments. If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever. Consequently, the standard libraries need to be very complete and very useful. And, while people seem to love the Rust approach to libraries, I'm not necessarily a fan. Far too many times I have pulled a library that is "obviously" something that a language should consider to be "standard library" and gotten bitten because it was broken. Only VERY core libraries in Rust are guaranteed to work across multiple architectures and OS's. I think Rust is probably doing the right thing for Rust as "batteries included" is NOT one of its tenets. However, that doesn't make it right for everybody else.
    • hn_throwaway_99 7y ago
      > If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever. Can you explain this more? What kind of place do you work? I've had some experience with large, bureaucratic companies, but nothing ever so far as "you can't install any other libraries."
      • dforrestwilson 7y ago
• stochastastic 7y ago
  The Python standard library has been a huge help for me. Evaluating which third party packages to trust and handling updates is a hassle. (Would love a solution for this. Does anyone have a curated version of PyPI?) I’m surprised that people want to slim it down other than for performance on a more constrained system. As an aside, why doesn’t the Python standard library extend/replace features with code from successful packages like Requests? Tried it and it didn’t work? Too much bloat? Already got too much on the to-do list?
  • llukas 7y ago
    +100 > As an aside, why doesn’t the Python standard library extend/replace features with code from successful packages like Requests? It is possible (ie. asyncio was separate package). It is slow process though.
    • quietbritishjim 7y ago
      pathlib is another example.
    • misterdoubt 7y ago
      I'd bet that absorbing Requests into the standard library, no matter the particular method of absorption proposed, would present too much of a political challenge to overcome.
  • jonnycomputer 7y ago
    I've come to use packages outside of the standard library very sparingly; been burned too many times to find that development of some package stopped or slowed down and backing out can be a real pita.
    • falcor84 7y ago
      What is your argument here? Standard library module development is also extremely slow.
• znpy 7y ago
  I have mixed feelings about this because I've seen both parts of the same situation. In certain situations, I've been working with a python interpreter on a RHEL machine where pip was not installed (and I was not allowed to install it as well as make other modification: the machine was owned by the client and I had to work with what I had available). - having some basic functionality in the core libraries was a godsend because I could work with that, even though it was not "ergonomic" - not being ergonomic, it was a "poor experience" (and certainly not optimized or anything nice to see).
  • deleted 7y ago
    [deleted]
• will4274 7y ago
  What's the point of saying that a standard library feature was not added soon enough? Nobody can go back in time and add it earlier. I can complain to death about features missing from C++11 or I can start using C++14.
  • pfranz 7y ago
    I think the point is that if those things in stdlib were external packages then older versions of the language would be easier to support because you could just update the package. I think the way it was phrased made it really easy to misunderstand. I think this goes to her point of Twisted wanting to support really old versions of Python and they would be a lot more comfortable not supporting really old versions of packages.
    • llukas 7y ago
      So they can keep python 2 on life support longer? That is not a good idea.
  • detaro 7y ago
    If that happens regularly, it makes stdlib less useful for other libraries to build upon, forcing them into (sometimes awkward) workarounds, and is a sign of the stdlib not getting enough maintenance.
• deleted 7y ago
  [deleted]
• wirrbel 7y ago
  Its not only that python stdlib libraries are getting old and sometimes appear to be unmaintained, some of the more recent additions and changes are lacking.
• raverbashing 7y ago
  Good points, some things on the standard library are just painful My "favourite" library quirk. socket.fromfd is only available on "Unix" on Python 2.X, that was fixed in Python 3.X The worse offender being the logging library. It's the least pythonic thing in the whole std library (ok maybe ABC is worse, but oh well)
• esotericn 7y ago
  Most of this post seems to revolve around the idea of py2 having an outdated standard library. https://pythonclock.org/ https://pythonclock.org/ has Py2 reaching EOL in 7 months. Realistically I'd say the time passed years ago. py3 is over ten years old now. We're not talking about some new unstable piece of kit, I'd imagine that a large percentage, perhaps even 50%, of the HN audience started their career after the transition had already started.
  • detaro 7y ago
    You read that wrong. Most of the points apply to Python 2 and 3 equally, or even only to Python 3. (obviously with the exception of the one of Python 2 not having gotten some useful bits, but the pattern continues)
• mattbillenstein 7y ago
  stdlib acts as the foundation upon which a lot of the 3rd party stuff us built - it's a feature to have it move slowly and not break often.
• nbAYT 7y ago
  I think it is important to notice here that a lot of the tension here is also Twisted vs. asyncio. I've never liked asyncio, while Twisted felt natural to me. So I would agree that an inferior solution has been pushed heavily in the stdlib and also to the syntax level. Moving the entire stdlib to PyPI is of course entirely foolish and would destroy Python.
• hermanradtke 7y ago
  > Brown went further adding that because few Python core developers are also major library maintainers, library authors’ complaints are devalued or ignored. PHP has a similar issue to this. The people writing C were not using the language. The best example is PDO. A lot of C was written, but it was essentially abandonware because the PHP users could not make any changes without getting the C maintainers to both agree and have the time.
• CodiePetersen 7y ago
  I like python, its a nice simple language that you can use to pump out a proof of concept real quick with little hassle, but for full on production I avoid it. But I think this is a larger trend in programming, in my opinion the majority of programmers are super lazy. Everyone is in a mad dash to get the cool new thing out so they just slap a bunch of dependencies on it and damn the consequences of developer debt down the road. More people would rather roll with an MVP as the final product than build something from scratch that's more robust, resilient, and efficient. Then after a while you have this huge mess of old broken code that can't be fixed anymore and just needs to be redone from scratch. Sure you are going to need to redo code anyways from scratch eventually. But, programmers like I mentioned, which is surprisingly a huge chunk, make problems worse for themselves throughout the lifetime of the code by being short sighted and stamping their approval on code their too lazy to rewrite because their boss doesn't know any better.
  • tomrod 7y ago
    > More people would rather roll with an MVP as the final product than build something from scratch that's more robust, resilient, and efficient. Well, yeah, of course. It's expensive to reinvent the wheel.
    • CodiePetersen 7y ago
      Its more expensive trying to fix broken code and working around other work arounds that should not be in production. Slows you down, bloats your code, makes it harder for new employees to learn the system when they come on board, etc.
      • Felz 7y ago
        Is the quality of the dependencies you use really so bad that it'd take you more time to fix them than to write your own code and then fix that? Like I can see where I'd make that tradeoff, but it'd have to be a small function in a really niche use case, which isn't really that common. (I work on the JVM though and don't know how good/bad Python libraries would be in general.)
• twblalock 7y ago
  The story of Python 2 to Python 3 migration, in a nutshell: > Van Rossum argued instead that if the Twisted team wants the ecosystem to evolve, they should stop supporting older Python versions and force users to upgrade. Brown acknowledged this point, but said half of Twisted users are still on Python 2 and it is difficult to abandon them. The debate at this point became personal for Van Rossum, and he left angrily.
  • someguydave 7y ago
    Hopefully the “python foundation” will declare python 2 deprecated soon so that it can be handed over to responsible maintainers.
    • Groxx 7y ago
      That's happening: https://pythonclock.org/ https://pythonclock.org/ To ensure things move along: pip has been printing highly-visible "python 2.7 will deprecate soon" warnings for a couple months or so now.
      • schlenk 7y ago
        And backing out of it when running on pypy, as that does not deprecate python 2 compatibility...
        • Groxx 7y ago
          Sure. Pypy is a separate implementation, they only control CPython. That's a pretty normal arrangement - official moves on, other forks might backport fixes for longer or focus on stability or some other realm of performance or something.
      • Izkata 7y ago
        Probably only on a recent pip version. Pip 10's dependency resolution doesn't like our requirements files (we have contradictory versions that work due to the order they are in the file), so we've mostly only gone up to pip 9.
• codr7 7y ago
  Isn't part of the issue mixing general purpose code that doesn't change very often (the kind that belongs in a standard library) with code that changes all the time (the kind that belongs on PyPI)? I remember one of Go's core devs voicing some of the same concerns regarding the SMTP-library [0] a while back. https://golang.org/pkg/net/smtp/ https://golang.org/pkg/net/smtp/
• deleted 7y ago
  [deleted]
• VectorLock 7y ago
  It seems weird to me that they specifically call out only Guido's behavior. I feel like there was probably a lot of context lost in "he left angrily" and its a bit unfair bordering on disingenuous.
• girlsrule1234 7y ago
  Some of her concerns do make sense, but the using, “a lot of our users stil use python 2.x” as a justification, in 2019, is ridiculous. Those same users had years to adopt/change the code base.
  • altmind 7y ago
    dont put all the blame on users. there's a ton of software that is python 2 only, for example gyp.
• sam0x17 7y ago
  A lot of this is side effects of the Python 2 vs 3 schism imo. If it weren't for that situation, practically everyone would on be 3.x, and supporting older versions wouldn't be important, so package maintainership wouldn't be as difficult. Put another way, the whole Python universe from my point of view has become a cautionary tale about breaking changes. Given Python's popularity, this might be an unpopular opinion, but I have yet to find someone who loves Python who still loves it as much when they discover other newer languages (I'm sure you exist, I just haven't met you!). Python is having its time in the sun really because it is a default install for most unix distributions, so even people stuck in government labs can use it because 2.7 is already installed. Even apt depends on it via the debian software-properties package, so it isn't going anywhere any time soon. The real question is how many people would use Python if it was as little known as, say, Elixir.
  • newen 7y ago
    Yep, Python is just a plain, boring interpreted language from the 80s that was designed as a reaction to Perl's syntax, and is unfortunate enough to contain a lot of dynamic properties that designing a JIT compiler for it is a massive amount of work. It can be seen in context with interpreted languages during that time such as Perl, Tcl, awk, sed, etc. It became popular because it was baby's first language taught in universities to both CS and non-CS majors. Python is simply not competitive in terms of language features with modern programming languages. I tend to assume most of the people praising Python are amateur or new programmers.
• banachtarski 7y ago
  I had to code in Python recently and had PTSD over all the syntactically significant whitespace. I have no idea how I ever found productivity in the language coming back to it now with fresher eyes. Refactoring, editing, and writing new code feels like such a drag.
  • mruts 7y ago
    I don't think Python is worth using for most new non-data science projects nowadays. The language is ugly and hard to work with. Also the performance is terrible.
    • mixmastamyk 7y ago
      There’s no accounting for poor taste.
• killjoywashere 7y ago
  > Standard Library Modules Crowd Out Innovation This heading is the essential problem in innovation writ large: some giant can ignore you and squash you without any effort at all, without even considering your existence.
• killjoywashere 7y ago
  At this point my MVP version of python is Anaconda-latest.
  • orbifold 7y ago
    For me it is Miniconda + a list of package requirements. Then you can just create an environment for each project and install new dependencies as you go.
• sametmax 7y ago
  Hawk Owl is a _fantastic_ dev. She was the main force behind the twisted 2->3 transition. But because she is, she is missing the point of batteries included. Asyncio is in the stdlib so that we have an official lib and API. The main benefit is that most people now, when looking for async, are not wondering about twisted or gevent or tornado. Most just go asyncio. Most dev efforts go to asyncio. It's the end of the great async war. Is it perfect ? No. And I don't care. It's one thing less to worry about. For those who know what they are doing, you can still choose and pip install twisted, but most people don't, and that's solved. Before that, just choosing the lib was a nighmare, as basically it's a definitive call. Out it on pypi, even with a "stdlib" tag, we go back to the 200X era. And it was not fun. And the goal for having things like xml/sqlite/ssl without installing anything makes python very useful in a load of situations where you can't install stuff. Sometime you are offline. Sometime you are in a restricted env. Sometime you are not on your machine. Sometime your security protocol is hell. Don't assume people use Python as we do, from our comfortable dev laptop driven by the knowledge of our craft. Python is used in banks, by scientists, in schools, by kids, by poor people in the third world, by geographers and pentesters. The python user base is incredibly diverse, it's why it's so popular: it fits a lot of use cases. So I see the benefit of having a side version of official modules we can pip install that can move faster. I see the benefit of cleaning the stdlib of old stuff, like the wave module, Template or @static. But I'm glad I don't have anything to install to generate a uuid or unzip stuff. I'm glad I don't have to worry about twisted anymore (depiste that I did write a book on the topic !). Also, pip install is NOT simple when you learn the language. I have to spend some time in the classroom, even with adult professionals, to explain the various subtleties of site-packages, import path, py -x on windows, python-pip on linux, -m, virtualenv, header files, etc. before my students become autonomous with it. Without a teachers, this turn into months of bad practices and frustrations. You'd have to fix that first, way, way before moving stuff to pypi. I do think it should be high priority actually: it affects way more than pip.
  • 1_player 7y ago
    Having a huge standard library also kills analysis paralysis and lets people be more productive. If you're in the flow and trying to hack together something, the last thing you need is to lose all momentum to pick a date time library. I've had this issue tons of times with Node and Rust, where I'm not up to date with the current meta and my 30 minute hack job is interrupted 5 minutes in by having to google which library should I use to do an HTTP request. (I've actually lost interest in whatever I was doing a few times because of this.) Python's stdlib is nobody's favourite, but when you start to get to its limits, you're probably past your flow state, you've written most of the logic and you can spend some time to replace http.client with requests because the latter is much better. On a tangent note, I've been trying to find another scripting language to replace Python because I'm not a fan of it anymore (I won't get into it right now), and considering what I just wrote, there's not much that can replace it, as most languages have a bare-bones standard library and if you're not up to date with the current best library to do X, you'll never achieve great productivity.
• i386 7y ago
  > The debate at this point became personal for Van Rossum, and he left angrily. Guido should stop acting like a child. Listening to people, hearing them out - even when it’s uncomfortable - is the mark of a good leader. I tell new PMs “this is the best job in the world 90% of the time but the other 10% is eating shit with a smile”
  • sametmax 7y ago
    He is not. He has been working on it for 25 years and he is just fed up to having to explain again and again the same things. He doesn't want to see the careful and long work he did being dragged to a standard he considers lower. There is always a new person coming with a new idea. It's exhausting, because it's required to make the language evolve, but it's also a new opportunity to screw things up evertime. And if he had let most people got their way during the last 2 decades, python would have ended just meh. Of course, everytime a new debate starts, everybody thinks that this time, just this time, he is wrong and they are right. I did too. We are all part of it. I get the reaction. There is a limit to what a person can take, and it's why he stepped down as a bdfl. But seing your baby and your reputation at stake is hard.
    • mruts 7y ago
      I dunno, Python is the epitome of a "meh" language. Guido has help Python back with his antiquated "get off my lawn" attitude since he created it. The language itself is inferior in expressiveness and performance to almost any other modern language. The only reason anyone uses it anymore is the network effects of the library are very strong, especially is fields relating to ML and data science.
      • sametmax 7y ago
        Those libs did not come out of nowhere, and the language did not rise from 1991 to the today without inherent qualities that draw people to it. Python didn't have any specialty like PHP, or an accidental monopoly like JS. It didn't come with a killer app like Ruby. It hasn't been made by a giant company like Go. It's pretty much a self-made language.
• 3327 7y ago
  Its never too late to rewrite python
• rmtech 7y ago
  There's an important tradeoff going on between library code that is (in theory) trusted and library code that is less trusted but has other advantages like solving a problem better or being a solution to problems that the most trust code can't solve. Right now the equilibrium in this tug-of-war is that a certain set of functionality comes by default in the python standard library and everything else is just a package that you can install. Obviously from a dev point of view it's a hassle to have to decide which of two or more packages for X is best, the pythonic way would be that there should be one and only one package for X. Of course at the cutting edge there have to be competing packages because there needs to be room for innovation. But obviously not everything can be in the python standard library. Not really sure what the solution is, but maybe there should be tiers of packages, with "Tier 1" being standard library, "Tier 2" having some kind of official stamp that it has been security audited to a certain standard, that Python has some control over who gets to modify it and why etc. Then maybe "Tier 3" could cover everything else, i.e. any bob random can go make a package on PyPi and it's Tier 3. In addition, the process of going from Tier 3 to Tier 2 would give people a chance to winnow libraries down to one way of doing each thing at the Tier 2 level. This might not be realistic but it's what my gut is telling me. C&C welcome.
• pm24601 7y ago
  I got lost on the "we have to back port to python 2.7" argument. Force the upgrade already. Code still on 2.7 if still useful can be upgraded.