25 ms·

Mercurial’s journey to and reflections on Python 3

• intrepidhero 7y ago
  > Matt knew that it would be years before the Python 3 port was either necessary or resulted in a meaningful return on investment (the value proposition of Python 3 has always been weak to Mercurial because Python 3 doesn't demonstrate a compelling advantage over Python 2 for our use case). What Matt was trying to do was minimize the externalized costs that a Python 3 port would inflict on the project. He correctly recognized that maintaining the existing product and supporting existing users was more important than a long-term bet in its infancy. Having just done transitions on a number of much smaller projects I had the same thought. Changes to string handling tripped me up and the changes to relative imports took some thinking. But the biggest frustration was the nagging question: Why am I doing this? edit: missing word
  • libria 7y ago
    > Why am [I] doing this? Lack of security updates past 2019 forced our hand. Did you find a way around that?
    • hsivonen 7y ago
      There's a project for keeping Python 2 alive: https://github.com/naftaliharris/tauthon https://github.com/naftaliharris/tauthon It's particularly uncool that Guido brought up the prospect of lawyers (https://github.com/naftaliharris/tauthon/issues/47#issuecomment-266470996 https://github.com/naftaliharris/tauthon/issues/47#issuecomm...) to force it not to be called Python and opposed to letting people who care about keeping Python 2 alive evolve it as "Python 2". (I know he has the legal right to insist on the name change. Still uncool.)
      • Klonoar 7y ago
        Longtime Python dev who was also annoyed by the 2 - 3 transition here. I don't see Guido as in the wrong for that. It'd be a smack in the face when you spend years trying to finally push people to switch (for better or for worse) and then a project like this takes the SEO and gets to run freely with it.
        • hsivonen 7y ago
          Why should Guido or PSF get to tell people to stop using Python 2 even if they no longer want to work on it? It's ungraceful not to hand off maintainership on good terms to someone who wants to do the work. Imagine if Stroustrup had done D and insisted that it be called C++ and wanted everyone to stop using the language everyone knew as C++ on Jan 1st 2020.
          • dragonwriter 7y ago
            > Why should Guido or PSF get to tell people to stop using Python 2 even if they no longer want to work on it? They aren't stopping people from using Python 2, the language or Python 2, the software. They are stopping people from using the name “Python” as the name of forked implementations of Python 2 not maintained by the PSF. No implementation not maintained by PSF is allowed to be called unqualified Python; the name is an important indicator of provenance. There are and have been plenty of third-party Python (2 and otherwise) implementations, the implementations just need their own names.
            • hsivonen 7y ago
              > They aren't stopping people from using Python 2, the language or Python 2, the software. The effort to claim the binary name python for Python 3 is actively hostile to Python 3 and a thing that runs unmodified Python 2 unmodified on the same operating system installation. (It's unclear to me how much this is a PSF push, but at least the PEP isn't telling distros to refrain from this hostile-to-comaptibility action.) > No implementation not maintained by PSF is allowed to be called unqualified Python The best situation would be PSF hosting continued Python 2-compatible development by people who want to do the work.
      • hackbinary 7y ago
        I disagree. If someone else wants to continue to develop Python 2 outside the Python foundation and formalised development community, then that is their prerogative, but Python has the right to decide what is and is not Python. Dilution of what is commonly accepted to be Python would not be a good thing, and would further add to confusion. I know that platform upgrades are painful, but we need to move with the times or we'll all be mired in technical debt and old technology.
        • takeda 7y ago
          Yes, them keeping Python 2 alive for 10 years where Python 3 was developed caused a lot of issues, it would be extremely short sighted to allow third (incompatible) python into the mix.
          • hsivonen 7y ago
            > it would be extremely short sighted to allow third (incompatible) python into the mix The whole point of Tauthon is that it is compatible with Python 2 (in the direction that old programs work).
      • mjw1007 7y ago
        I think it makes a great deal of sense for the Python core team to say "we're finished with Python 2 and want nothing more to do with it". But I'm very disappointed that the Python Software Foundation isn't explictly supporting people who want to keep Python 2 compiling and running on modern systems. I think that would be well within their remit to "promote, protect, and advance the Python programming language". This is particularly so because Python is widely used for scientific purposes, and being able to reproduce old results is valuable. Even before Python 3.0 appeared, I came across scientists saying "I prefer to stick with Fortran because new Python versions break old code too frequently".
        • int_19h 7y ago
          PSF does not object to people who keep Python 2 compiling and running, such as ActiveState (https://www.activestate.com/company/press/press-releases/activestate-offers-extended-support-for-python-2-beyond-eol/ https://www.activestate.com/company/press/press-releases/act...). This case is different, because it's a project that uses the Python name, but actively adds features to the language. This is the classic example of brand confusion - someone might try to use it, find something to complain about, and PSF's reputation suffers as the result. They also get support overhead from the users of the fork (even if all they do is tell them to go away, that is still triage time that could be spent on other issues).
          • mjw1007 7y ago
            "Does not object" is better than nothing, but I think it would be better if the PSF actively helped to coordinate this work (again, without bothering the core team). As far as I'm concerned, this is exactly the sort of thing that the PSF exists for.
        • forgotpwd16 7y ago
          >This is particularly so because Python is widely used for scientific purposes, and being able to reproduce old results is valuable. You can always download an old version and the respective libraries and use them to reproduce any results you want. That doesn't mean that old version should be supported anymore.
      • Shorel 7y ago
        I downgraded Deluge to the Python 2 version because the new one doesn't work in Windows and I use both operating systems.
      • simias 7y ago
        I understand your point of view but on the other hand we can make the parallel with Perl 5 and 6. Having incompatible forks of the language share the same name is a pain for everyone involved. I can completely understand the "mainline" python maintainers not wanting to have to deal with that. Besides if the Tauthon people are serious about maintaining their fork long term it needs to become more than a mere fork and a real language ecosystem of its own, in the long run having a different name will probably help with that, assuming that they ever get there. EDIT: Also reading the rest of the thread I realize that the post that you linked out of context is slightly misleading (but I blame github's aggressive folding more than you here). Guido's answer comes after the following exchange: stefantalpalaru: "Disregard Guido's objection. The "Python" trademark doesn't extend to "py2" or "py28". Read this for details: https://www.python.org/psf/trademarks/" https://www.python.org/psf/trademarks/" Guido: "Isn't the whole point that we're trying to solve this without lawyers?" stefantalpalaru: "The whole point is that you've been sabotaging Python 2 for years and when someone does what needed to be done from the start, you come up with silly objections." Guido: "OK, bring in the lawyers." In that light, and given the other poster's ridiculously inflammatory take, Guido's answer seems rather level headed and appropriate IMO. He stands his ground, so to speak.
        • lizmat 7y ago
          Re: I understand your point of view but on the other hand we can make the parallel with Perl 5 and 6. Please note that Perl 6 has been renamed to Raku (https://raku.org https://raku.org using the #rakulang tag on social media). So Perl and Raku are now considered to be different languages, albeit from the same inspiration. Now, if Python 2 people would decide to rename Python 2 to something else, I guess it would be a mirrored parallel :-)
          • simias 7y ago
            That's precisely what's happening with the third-party Python2 forks. The renaming of Perl 6 occurred last October, specifically because of the problems caused by the confusion between the incompatible Perl 5 and 6 that caused a lot of trouble to the Perl people on either side for many years. It's not a mirrored parallel, it's the Python folks learning from Perl's mistakes and making sure that this parallel won't come to be.
      • camgunz 7y ago
        Letting "Python 2" zombie around is unacceptable. Python 3 is better in every way, and has been since 3.3 (which Armin deserves a lot of credit for). Consider anyone who wants to build something with Python, whether it's a library, application, or service. What's better, having to build for Python 3 and 2, or just Python 3? Thank God that Guido did this, despite knowing all the blowback he'd get. To me, that's super cool.
        • eesmith 7y ago
          "better in every way" ... except for 1) startup time (according to the linked-to article), 2) support for existing Python 2 code, and 3) support for Python 2 C extensions. For example, https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/ https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stabili... describes the "Molecular Modelling Toolkit (MMTK), which might well be the oldest domain-specific library of the SciPy ecosystem, will probably go away after 2020. Porting it to Python 3 is possible, of course, but an enormous effort (some details are in this Twitter thread[1]) for which resources (funding plus competent staff) are very difficult to find." [1] The thread at https://twitter.com/khinsen/status/930749714567434240 https://twitter.com/khinsen/status/930749714567434240 includes "Lots of C modules written for Python 1.4 are waiting for enthusiastic code archeologists ;-)". I don't think Hinsen is alone in that situation. I can well believe there are some people who, for example, plan to retire in about 5 years and would rather keep with with a Python 2 zombie than spend time to port working code to Python 3.
          • camgunz 7y ago
            Startup time is complex, but base startup only increased about 20ms, and that's being generous. I'll admit Python 3 is still slower at a lot of things. But that feels like saying your new dog is even worse at math than your old one. The C extension thing isn't Python's fault. It's the job of library and app authors to update. Do we complain that Vulkan has bad SunOS support? This is totally backwards. Could Hinsen (and others) not just version their deps? It's not like people are erasing Python 2 off the internet. If his main worry is reproducibility, he should be doing that anyway. --- I don't want to give the impression I like the whole Python 3 thing. I think it was a pretty big mistake and a huge missed opportunity. I'm very sympathetic to people who had to put in a lot of work for basically no good reason--Python 3 didn't really offer anything significantly better than 2 until... 3.5 (3.4 if you think the first pass at async was useful, I personally don't). But I also find the ballyhooing about it really insufferable. Yeah it was a mistake; Armin Ronacher (as usual) was right. It was also over 11 years ago. Time to forget all about this and build cool stuff, please please please.
      • MiroF 7y ago
        Did you see the other replies on the thread? Guido has absolutely every right here.
      • stefantalpalaru 7y ago
        > It's particularly uncool that Guido brought up the prospect of lawyers [...] to force it not to be called Python It's much worse. The naming proposals were "py2" and "py28" - names for which they had no trademark registered. That made it clear that it was never about trademark protection. https://github.com/naftaliharris/tauthon/issues/47#issuecomment-266314206 https://github.com/naftaliharris/tauthon/issues/47#issuecomm...
    • falcolas 7y ago
      > Lack of security updates past 2019 forced our hand. Did you find a way around that? Amazon is maintaining Python 2 for at least 4 years, as part of their Amazon Linux long term support release. Google app engine will support Python 2 for an unknown amount of time; they haven't announced an end date. PyPy is Python 2, with (to the best of my limited knowledge) no plans to deprecate support. There are also other LTS releases out there which include Python 2 support. IOW, the forcing function of the PSF no longer supporting Python is not as big a factor as was hoped.
      • viraptor 7y ago
        This would only help the server side of mercurial though. There's no client-side supported distribution really. Pypy is not that popular yet.
        • falcolas 7y ago
          I don't know about others, but when I used Mercurial, it was via installing it through brew. And if brew installed pypy as a dependency so Mercurial could still use Python 2, I probably wouldn't have noticed.
          • viraptor 7y ago
            You'd notice, because it wouldn't work: https://www.mercurial-scm.org/wiki/PyPyPlan https://www.mercurial-scm.org/wiki/PyPyPlan
      • rst 7y ago
        Security updates in Python itself aren't the only issue; a Python 2 project may also depend on packages with security issues of their own, which require continued upstream maintenance. For example, the python-saml package (for managing SAML-based single sign-on) has separate Python 2 and Python 3 versions, and implements a security-sensitive protocol which means it has (in the fairly recent past) gotten security updates for issues serious enough to rate an assigned CVE. If you're using it, having the current maintainers walk away from the Python 2 version is a serious risk...
    • kick 7y ago
      PyPy is keeping Python 2 support indefinitely, I believe.
• nemothekid 7y ago
  > Perhaps my least favorite feature of Python 3 is its insistence that the world is Unicode. [..] However, the approach of assuming the world is Unicode is flat out wrong and has significant implications for systems level applications (like version control tools). Isn't this more a problem with Python not easily differentiating between String and Byte types? Both Go and Rust ("""systems""" level languages) have decided that "utf-8 ought to be enough for anybody" and that seems to be a good decision.
  • steveklabnik 7y ago
    The blog post talks about this a bit in Rust, but we don't actually say that. We do make that the default, but we also give you the ability to get at the underlying things as well. There's a lot of interesting work here, actually, like WTF-8...
    • kevingadd 7y ago
      In the wild WTF-8 and its 16-bit equivalent show up more often than you'd expect. I ended up discovering a case recently where part of the .NET executable file format is actually encoding strings as WTF-16 (not UTF-16) and any internal lowering needs to map them to WTF-8 instead of UTF-8. Until that point I had expected to only ever encounter WTF-8 in web browsers!
  • dilap 7y ago
    A go string is just a sequence of bytes, which is usually/by convention utf8. But you can store anything you want in there, if necessary.
  • Jasper_ 7y ago
    Yes, but that insistence that Bytes and Unicode are two different things that Shall Not Be Mixed was mostly a Python 3-ism. Python 2 had different types but you could be sloppy and it would kinda work out. There was this assumption that Unicode code points were the correct single unit to talk about Unicode. You iterate over code points, you talk about string lengths in terms of code points, you slice in terms of code points. Much like the infamy of 16-bit Unicode, this is an assumption that has kinda gotten worse over time. Now we can and do want to talk about bytes, code points, and newer sets like extended grapheme clusters. I think this is probably the big failing of Python 3's Unicode model. Making a string type operate on extended grapheme clusters might fix it, but we'd be in for the same sort of pain, and the flexibility of "everything is bytes, we can iterate over it differently" of Go and Rust is much nicer in comparison. The second thing was this assumption that everything remotely looking like text was Unicode, despite this maybe not being true. HTTP has parts that look like plain text, like "GET" and "POST" and the headers like "Content-Type: text/html". But the correct way to view this as ASCII bytes, and no other encoding makes sense; binary data intermixed with "plain text" definitely happens, and the need to pick and choose between either Unicode or Bytes caused major damage in the standard library which still persists to this day -- some parts definitely chose the wrong side. Take a look at the craziness in the "zipfile" module for one other example. It's probably fixed now, but back then, I basically had to rewrite it from scratch in one of my other projects. They eventually relented and added back a lot of the conveniences to blur the line between bytes and unicode again, like adding the % formatting operator for bytes, which I think shows that their insistence on separating the two didn't really pan out in practice. And yet, migration is still a pain.
• weberc2 7y ago
  > One of the biggest early hurdles in our porting effort was how to overcome the string literals type mismatch between Python 2 and 3. In Python 2, a '' string literal is a sequence of bytes. In Python 3, a '' string literal is a sequence of Unicode code points. These are fundamentally different types. And in Mercurial's code base, most of our string types are binary by design: use of a Unicode based str for representing data is flat out wrong for our use case. We knew that Mercurial would need to eventually switch many string literals from '' to b'' to preserve type compatibility. But doing so would be problematic. > In the early days of Mercurial's Python 3 port in 2015, Mercurial's project maintainer (Matt Mackall) set a ground rule that the Python 3 port shouldn't overly disrupt others: he wanted the Python 3 port to more or less happen in the background and not require every developer to be aware of Python 3's low-level behavior in order to get work done on the existing Python 2 code base. This may seem like a questionable decision (and I probably disagreed with him to some extent at the time because I was doing Python 3 porting work and the decision constrained this work). But it was the correct decision. Matt knew that it would be years before the Python 3 port was either necessary or resulted in a meaningful return on investment (the value proposition of Python 3 has always been weak to Mercurial because Python 3 doesn't demonstrate a compelling advantage over Python 2 for our use case). As a general rule, this seems like good practice, but surely b-strings, print_function, etc are a trivial upfront cost, and one that would have to be paid sooner or later anyway?
  • kingemer 7y ago
    It sounds like the cost was non trivial for them, partially because they weren’t allowed to break things for python2, or even disrupt the efforts of those using it. The language wasn’t ready for the transition, but it feels like it may have been even harder on them because of the requirements imposed on their project.
    • kingemer 7y ago
      Considering how much opposition there is in moving to python3, has there been any significant effort in the community keeping python2 alive?
• peatmoss 7y ago
  I think my takeaway lesson is that it’s very hard to introduce large, breaking changes to a language and not alienate a large proportion of existing users. I don’t know that there’s a right way. I look at Perl, which was a juggernaut when I first used Python, and announcements of Perl 6 certainly didn’t help Perl’s slide. Often cited is the fact that Perl 6 is a totally different language unrelated by anything but creator and name. The Perl brand was not enough to carry the bulk of Perl users from Perl 5 to Perl 6. Perl 6 is now called Raku, which probably better reflects the magnitude of the change. On the other hand Python 3 is a small but still significant departure from Python 2. If they’d called Python 3 something else, we’d probably be griping about how superficially different from Python 2 it was without bringing substantially new ideas. Oddly my feeling is that Racket, in its departure from mainline Scheme, largely did retain its core audience, but that may have been a feature of its usage in academia. Fast forward to last year when a prominent Racket architect announced “Racket 2” which would completely change the syntax of the language. Prominent community members reacted negatively, due to fears of Perl 6’s fate. But now they’ve decided to simply call the new research language Rhombus and have reiterated plans to continue supporting Racket. I went from feeling very negative to the change to being okay with the direction. I’m not sure there are lessons to draw, other than noting than version bumping versus making a new language with a new name can be bad for entirely different reasons.
  • tus88 7y ago
    > introduce large, breaking changes without major benefits FTFY
  • mark_l_watson 7y ago
    I totally agree with you, I am uncomfortable with Racket 2 having a non-Lisp syntax. As someone who has used Lisp languages to get stuff done for over 30 years, I would say NO to the syntax change. That said, Racket is open source the maintainers have good reasons for a change based on getting a larger user base. I wish them great success.
    • peatmoss 7y ago
      I disliked the “Racket2” name because it strongly implied that Racket 1 (the one with parentheses on the outside and few commas) was the past, and that something with a vaguely Algol-ish syntax was the future. But Racket as a project was always about language experiments. I certainly didn’t bristle at Typed Racket or any of the other languages. And so, in changing “Racket 2” to “Rhombus” and committing to mainline support of Racket, I feel pretty comfortable with the direction. I find this fascinating that I feel this way given that nothing has really changed but the name.
• makecheck 7y ago
  It’s funny, on the Mac one becomes used to constant changes, rewriting damn near everything just to stand still. Yet I designed my Mac app long ago to depend on the system “Python 2” (bound to C++), because it seemed that both the installation itself and the Python language and libraries were very stable. Looking back, this turned out to be sustainable for a remarkably long time, as “Python 2” really did evolve only additively and there was almost no reason to even touch 15-year-old code that was relying on Python 2. For the Mac platform especially, this reliability is unheard of. More amazing to me is that in Catalina, the release famous for breaking just about everything else, “Python 2” is still there and works as it always has! Of course, Apple did announce that it will be ripped out in the next release. :)
  • pfranz 7y ago
    I think this weird thing happened with Python 2. I believe Python 2.6 (Oct-2008) was the last "feature release" and 2.7 (Jul-2010) was intended as a bridge. So since 2008, 2.x users have been shielded from most all of the normal churn of any widely used language that's in active development. What I don't think people realize is that not only are you expected to move to 3.x, but you'll have to keep up or fall behind with new 3.x releases. During that same period (since 2008) 3.x has had 9 big releases. Of course that 2.x stability was done with the assumption you'd move to 3.x and isn't sustainable for PSF indefinitely.
  • cannam 7y ago
    > Of course, Apple did announce that it will be ripped out in the next release They did? Damn, I was using that...
• ufov2 7y ago
  The approach of doing the transition slowly over many years maybe was a mistake here, and another thing making it harder seems to have been little support from the top of the project. I ported two projects with ~200000 Python-SLOC (about the same size as Mercurial according to sloccount) back in the early 3.x days. Doing this via more or less flag-day conversions within a few months, converting the codebases first to 2to3-able subset, and as a second step later on dropping 2to3 via common dialect of Python 2/3 with six, was not very painful in the end.
  • sfink 7y ago
    Did you have a large ecosystem of third party extension modules that also had to obey the flag day?
  • masklinn 7y ago
    > I ported two projects with ~200000 Python-SLOC (about the same size as Mercurial according to sloccount) back in the early 3.x days. Doing this via more or less flag-day conversions within a few months, converting the codebases first to 2to3-able subset, and as a second step later on dropping 2to3 via common dialect of Python 2/3 with six, was not very painful in the end. Sounds like you used the same method, just over a smaller timeframe: convert to a common 2/3 subset, then drop Python 2 at some later point.
• sprash 7y ago
  The transition from Python 2 to 3 was one worst things that could happened to the whole community. The costs of the transition never justified the benefits. The new features were negligible at best or a regression at worst and in some cases performance got even worse. One could even assume flat out sabotage. Let's just hope there will never be a Python 4 and the developers now finally start focusing on the greatest flaw of Python: performance.
  • Tanooki_Mario 7y ago
    What's crazy is if they had removed the GIL python 3 adoption would have been huge. All python 3 offered were marginal benefits for adopting functionality that broke huge code bases.
• michaelhoffman 7y ago
  The biggest problem with the Python 2 to Python 3 transition was not that breaking changes were made. It’s that breaking changes were made in a way such that you could not easily have code that worked both on Python 2 and Python 3. It took years before the advent of six, Python 3 u’’ literals, and modernize. The author discusses this at length.
  • j88439h84 7y ago
    Six was available for years (2011) before Mercurial even started porting (2015). https://github.com/benjaminp/six/graphs/contributors https://github.com/benjaminp/six/graphs/contributors
    • eesmith 7y ago
      That was part of the "discusses this at length". Part of the relevant discussion is: > So I'm not sure six would have saved enough effort to justify the baggage of integrating a 3rd party package into Mercurial. (When Mercurial accepts a 3rd party package, downstream packagers like Debian get all hot and bothered and end up making questionable patches to our source code. So we prefer to minimize the surface area for problems by minimizing dependencies on 3rd party packages.)
  • choppaface 7y ago
    Another big problem is there was no significant incentive to adopt Python 3. That’s why it took so long for large projects to transition. In comparison, during the last decade, C++ went from dodgy C++11 toy projects to all new code being written in modern C++. The modern feature set is that good.
    • Jasper_ 7y ago
      C++ doesn't mandate you switch from std::cout to fmt in order to use lambdas. If they did that, I think we'd see a lot less modern C++.
      • 7y ago
• fireattack 7y ago
  I understand author's reasoning in the context of a transition, but as a "non-Latin" language user, defaulting str to unicode literals is the best change in Python 3. Coming from C#, I never get used to Python 2's approach. It's a pain in the ass working with non-Latin characters in Py2 starting from simply output in console, especially on Windows. >assuming the world is Unicode is flat out wrong True, but Py2's approach makes lots of developers assume the world is Latin-1. I see way too many examples of things broken on a Chinese locale environment, including Python's official IDLE ([1]). [1] https://bugs.python.org/issue15809 https://bugs.python.org/issue15809 (Summary of this bug: in 2.x IDLE, an explicit unicode literal used to still be encoded using system's ANSI encoding instead of, well, unicode.)
  • earthboundkid 7y ago
    Python 2's approach was bad, no argument, but the transition plan for 2-to-3 just didn't work. They thought everyone would run 2to3 in a big bang, and then we'd all switch over to 3 in a few years. Instead it dragged out over a decade because in reality we needed to write code that was compatible with both 2 and 3 (the "6" approach) until enough things were on 3 to drop 2 support. Hindsight is 20/20 naturally, but in retrospect, they should have just made `bytes` into the name for old `str` and used `from __future__ import` to create a gradual system for moving from 2 to 3 instead of a big bang "we'll break everything once and then never again".
    • deleted 7y ago
      [deleted]
    • cbsmith 7y ago
      I'm not sure they really thought 2to3 would be used for a big bang. I seem to recall the general initial messaging was that Python 3 was a new language and you would need to do a language port to get to it.
  • int_19h 7y ago
    The most amusing quote in the entire article is this (emphasis mine): > This ground rule meant that a mass insertion of b'' prefixes everywhere was not desirable, as that would require developers to think about whether a type was a bytes or str, a distinction they didn't have to worry about on Python 2 because we practically never used the Unicode-based string type in Mercurial. Requiring developers to think which one it should be is, of course, the whole point of the changes in Python 3 - and it's what produces better apps that are more aware of i18n issues in general and Unicode in particular. And the complaint doesn't even make sense if taken at face value - if all strings in Mercurial are byte strings, then what is there to think about? just use b'' throughout, no need to worry about anything else. Of course, the devil is in the details, which is reflected by the word "practically" in that sentence - this kinda implies that there are places where Unicode strings are used. At which point you do want the developers to think about bytes vs Unicode. So the real complaint is that Python switched the defaults in a way that made bytes-centric code more complicated - because it has to be explicit now, instead of the Python 2 world, where bytes was the default, and Unicode had to be requested explicitly. Which, of course, is the right change for the vast majority of code out there, that operates on higher level of abstraction, where "all strings are Unicode by default" is a perfectly reasonable assumption to force.
• deleted 7y ago
  [deleted]
• j88439h84 7y ago
  A lot of Mercurial's issues would have been resolved much easier if they'd used the common tools for maintaining polyglot 2/3 code instead of trying to invent everything themselves. Futurize and Pasturize in particular provide essentially all of the features that this post laments missing. https://python-future.org/ https://python-future.org/
  • lacker 7y ago
    The author does touch on that. When Mercurial accepts a 3rd party package, downstream packagers like Debian get all hot and bothered and end up making questionable patches to our source code. Some environments just can't use dependencies like this. IMO Python 3 was too much of a breaking change, and in particular, the ability to transition from 2 to 3 should have been better in Python itself.
    • j88439h84 7y ago
      Six, for example, is designed to be a single file -- specifically to ease copying it directly into the code base. But the idea that Mercurial couldn't use dependencies because of fear for what Debian might do...I find it so hard to believe that's the best choice. Vendor if you must, but do not reinvent the wheel.
      • masklinn 7y ago
        Having done so, six really is not necessary for a transition. We went with forking werkzeug's minimalist compat file because six was waaay overkill, and that made it easier to progressively rip it out later. python-future is invaluable for the fixers (which are way better than 2to3's), the runtime stuff you can easily do without.
  • untitaker_ 7y ago
    Those tools were not popular at the time for sure. I remember that Python-Future only got traction sometime after Flask has been ported to Python 3.
• alangpierce 7y ago
  It's interesting that they wanted to add b' prefixes to all strings, and I wonder if they would have had a better experience by embracing regular strings instead. At least in Python 3, if your string only contains ASCII, then the underlying representation will use one byte per character, so ASCII-only strings are stored just as efficiently as ASCII-only bytes instances. I think there are two mental models for how to approach the str/bytes split: 1.) A `str` is for unicode use cases, and a `bytes` is better for cases that don't support unicode. 2.) A `bytes` is an array of numbers between 0 and 255. A `str` should almost always be used when your value is conceptually a sequence of characters. `str` doesn't imply that arbitrary unicode is allowed, and it's fine to have a convention that a particular `str` is ASCII-only, just like other conventions you might have on variable values. My impression is that #1 is the Python 2 mental model and is tempting for Python 3, but that #2 often works better when writing Python 3 code. Under mental model #2, asking for "%s" formatting is really asking for a replacement strategy that detects the number 37 followed by the number 155 in an array of numbers and fills in a sub-array, which seems more strange and likely to get false positives if you're really working with binary data like the bytes of a .jpg file. That said, I'm sure the devil is in the details, and maybe a project like mercurial has to stay backcompat with bytes data that is neither ASCII nor valid UTF-8, or some other compelling reason to stick with bytes everywhere.
  • CJefferson 7y ago
    The problem is many strings might contain things like commit messages, or filenames, neither of which has to be valid unicode. I've had the same problem with a few Python 2 -> 3 conversions -- everything is fine until you have to operate on text or filenames which aren't valid utf8/unicode.
    • alangpierce 7y ago
      Got it. So I understand, maybe someone saved a filename as the latin-1 encoding of some non-ASCII text, and Mercurial would need to support such files (but also would have no contextual information that it's latin-1)? I'm tempted to say "nobody should have filenames like that", but I guess a project like Mercurial needs to be as compatible as possible. Are there modern use cases for filenames like that, or is it fair to say it's all legacy data?
• rossdavidh 7y ago
  Having worked in python for about a decade, first in python2 and lately in python3, and having seen projects convert, I find this article baffling. I found Six to work pretty well, and where it didn't it wasn't hard to change. I think the core error here was in NOT doing what he calls a "flag day" conversion. Sometimes it is easier to do something quickly, than to live with it happening slowly. I've done "flag day" conversions, and they were pretty painless, if stressful at the time.
  • klodolph 7y ago
    I think Six works pretty well for a certain percentage of projects, and then completely blows up for others. Same thing applies to using Mypy. Some modules are easy to add annotations for, other modules have insanely complicated types.
  • the_mitsuhiko 7y ago
    > Having worked in python for about a decade, first in python2 and lately in python3, and having seen projects convert, I find this article baffling. I found Six to work pretty well, and where it didn't it wasn't hard to change. It matters a lot where you work. If you are in high level land Python 3 is not much of a chance. If you work at the boundary (wire protocols, OS interop, text transformation) then Python 3 is a significant step back, especially before 3.6. A lot of the mud that Mercurial stepped through is also where I went through with my libraries. The day I managed to get the PEP through that reintroduced the u prefix on strings was also the last time I voluntarily participated in a language summit. The atmosphere was awful and not evidence based.
    • Jasper_ 7y ago
      Yeah, I was around for the sidelines of PEP-461, reintroducing % onto the bytes type, trying to get things done behind the scenes, and it was just a miserable experience all around. I don't think anybody cared to understand our concerns about why we should bother making the bytes type useful. At times it seemed like they believed leaving in the concatenation on the bytes type was a mistake. Read through the receipts here: https://bugs.python.org/issue3982 https://bugs.python.org/issue3982
• loxs 7y ago
  It would probably be less painful and much better (for other reasons) to migrate to some other language. Some projects did that successfully, or are in the process of doing so. Most notably reposurgeon: https://gitlab.com/esr/reposurgeon https://gitlab.com/esr/reposurgeon
  • sfink 7y ago
    Greg says as much in the article: in hindsight, porting to Rust would have worked out better. Which is a pretty bold statement, but very interesting to hear from someone with intimate experience to back the opinion up.
    • cookiecaper 7y ago
      Mercurial's dependence on Python has always held it back, IMO. Self-contained Rust or Go-style static binaries work much better for "install everywhere" system utilities. I'd love to see Hg port to a more concise ecosystem and potentially claw some of the market away from Git.
      • mixmastamyk 7y ago
        The author wrote pyoxidizer for that purpose.
    • ufmace 7y ago
      In a sense, Rust may have been a better choice for Mercurial overall, but it's hard to imagine how much of a pain the migration process would be. I don't think you could make much of any headway going Python 2 -> Rust with automated tools. That means the transition would look like, stop all Mercurial dev in its tracks, have all current contributors (who can and care to) learn Rust, bring on a couple of devs with experience in architecting large Rust projects, spend however long redesigning and rewriting in Rust, release a roughly feature-equal version a year or 2 later. Good way to move Mercurial from second-place to Git to barely known.
      • cesarb 7y ago
• raverbashing 7y ago
  Ok, yeah, maybe mercurial's case was special, but still, this seems like they made it harder on themselves needlessly. > One was that the added b characters would cause a lot of lines to grow beyond our length limits and we'd have to reformat code ORLY?! Well, guess what: hard line size limits are stupid. Now you know why. That's why "foolish consistency is the hobgoblin of little minds" is one of the 1st phrases of PEP-8. But I'm tired of people saying "oooh let's cut all lines to be under 80-characters" like it's some kind of Biblical Mandate. No, it isn't. And the 80 chars limit is BS. Probably the part I hate the most about PEP-8 (and especially how people interpret the PEP-8) > is its insistence that the world is Unicode Oh please. Yes, the world is Unicode. Get over it. Maybe not bytes on disk/network. But apart from that? Yes. If libraries take bytes or unicodes I can agree it's a thorny issue, but let's move on because a happy day is a day where I don't get an UnicodeDecodeError because Python2, to add insult to the injury thinks the world is not only not Unicode, but it's all ASCII. Windows made the right call a long time ago when it decided to make all strings Unicode. Ok, maybe UTF-8 would be better than 16, but it still does the job. But I have to agree with them that any version < Py3.4 or 3.5 was really not worth it.
  • yjftsjthsd-h 7y ago
    > the world is Unicode. Get over it. Maybe not bytes on disk/network. But apart from that? Yes So, just ignoring the 2 things that a version control system exists to work with directly?
    • raverbashing 7y ago
      You're ignoring the parts where they have several hardcoded byte/unicode strings. Otherwise just convert to and from when saving and sending it to network
• ploxiln 7y ago
  I've been involved in multiple non-trivial libraries and frameworks that supported both python2 and python3 for many years with the same codebase ... and it really wasn't anything like this. The python3 "adaptation" effort for mercurial was just bungled by multiple terrible decisions. First was the idea that normal feature contributors should not see any b"" or any sign of python3 support for the first couple years of the effort. Huge mistake. You need some b"". But you don't need all b"" everywhere. That was the second huge mistake. Don't just convert every natural string in the whole codebase to b"". The natural string type is the right type in many places, both for python2 (bytes-like) and python3 (unicode-like). The helpers for converting kwargs keys to/from bytes is a sign that you are way off track. This guy got really hung up on the fact that the python2 natural string type is bytes-like, and tryied to force explicit bytes everywhere (dict keys, http headers, etc) and was really tilting at windmills for most of these past 5 years. Yes, you pretty much had to wait for python-3.4 to be released and for python-2.6 to be mostly retired in favor of python-2.7. Then, starting in early 2014, it was pretty straightforward to make a clean codebase compatible with python-2.7 and python-3.4+, and I saw it done for Tornado, paramiko, and a few other smaller projects.
  • CJefferson 7y ago
    Interesting you mention http headers. I had a program converted Python 2 -> Python 2 which was crashing occasionally, and it turned out it was because I was being sent a http request which wasn't valid unicode, so decoding failed. I had to switch back to treating headers as bytes for as long as possible. It is a stupid client which doesn't send valid ascii for http headers of course.
    • dnautics 7y ago
      > It is a stupid client which doesn't send valid ascii for http headers of course. ...or a smart malicious actor.
    • takeda 7y ago
      I believe the headers are encoded using ISO-8859-1 not Unicode. That encoding has 1:1 mapping with bytes so wouldn't break this way. Treating them as UTF-8 was the bug.
• deleted 7y ago
  [deleted]
• reggieband 7y ago
  In the past I have been a vocal advocate for the way the transition from Python 2 to Python 3 was handled. However, it should be said I use Python primarily as scripting glue, e.g. for build scripts and automation tasks. I have never worked on a "large" Python code base nor did I have to migrate anything. Almost everything I had written in Python 2 was just naturally replaced by newer scripts in the due course of time. I also remember my first forays into Python 3 and the annoyance I had at some of the decisions. I recall when they relented on the % operator for string interpolation and I agree it was a poor initial choice to leave it out. I totally agree with the author that Python 3 could have made some subtle changes earlier on to help those with massive codebases. And I still feel it was the right move. Somehow Python is even more relevant today than it was when this painful process began. While some may say that popularity is despite missteps I actually believe the general slow and cautious push forward is one of the primary reasons Python continues to succeed. There is a balance between completely abandoning old users (e.g. Perl 5 to Perl 6) and keeping every historical wart (e.g. C++). IMO, the Python community found a middle ground and made it work.
  • lizmat 7y ago
    Re: There is a balance between completely abandoning old users (e.g. Perl 5 to Perl 6) Please note that Perl 6 has been renamed to Raku (https://raku.org https://raku.org using the #rakulang tag on social media). In the original design of Perl 6, a source-level compatibility layer ("use v5") was envisioned that should allow Perl 5 source code to run inside of Perl 6. So the plan was to actually not abandon old users. In my opinion, this failed for two reasons: 1. Most of Perl 5 actually depends on XS code, the hastily devised and not very well thought out interface to C code of Perl 5. Being able to run Perl 5 source code in Perl 6 doesn't bring you much, unless you have a complete stack free of XS. Although some people tried to achieve that (with many PurePerl initiatives), this really never materialized. 2. Then when the Inline::Perl5 module came along, allowing seamless integration of a Perl 5 interpreter inside Perl 6, using Perl 5 modules inside of Perl 6 as if they were Perl 6 modules, it basically nailed the coffin in which the "use v5" initiative found itself already in. And now they're considered different languages after the rename to Raku, dividing already limited resources. I guess that's the way of life.
• ascotan 7y ago
  Python 3 is the new Windows Vista.
  • marcosdumay 7y ago
    You mean they just have to fix the issues behind the scenes, then rename the last version (like to "Python 4"), and it will become the greatest version ever?
    • yjftsjthsd-h 7y ago
      Yes, actually. Now that we've gotten through the pain of the first years of Python 3, if we could have a clean start and call Python 3.8 Python4, it would probably be well-received.
• war1025 7y ago
  We are on the brink of completing the transition to python3 at my work. The end result of this is that I just spent a good chunk of last week reviewing a pull request with 70,000 lines of changes, which was one of the final in a series of ~10k line pull requests that came in through the fall. All of this was the heroic effort of one of my coworkers who had the unenviable task of combing through our entire codebase to determine "This is unicode. This is bytes. Here is an api boundary where we need to encode / decode." etc. It was a nightmare of effort that I'm glad to have behind us.
  • vmchale 7y ago
    > All of this was the heroic effort of one of my coworkers who had the unenviable task of combing through our entire codebase to determine "This is unicode. This is bytes. Dynamic typing!
    • war1025 7y ago
      Not dynamic typing's fault. The issue is they changed the types out from underneath you. And then left it to each library to decide which type it was actually going to accept.
      • magicalhippo 7y ago
        But with static typing, the compiler can let you know when you're doing something wrong with the new Unicode-based string.
        • war1025 7y ago
          Well, it can, except you then need to go through and update all of your internal APIs to be correct. Really the string transition was just a poor choice in my opinion. Python2 already had unicode strings that were easy enough to specify (just prefix with a `u`). It would have been better to just delineate that barrier better from an API standpoint. I understand the appeal of having unicode for the default string literal type, but it was actively hostile to existing projects.
• mikl 7y ago
  There’s no doubt that the 2 -> 3 transition was rough for the Python community. I personally stopped using Python as my go-to language in the early Python 3, since writing Python 2 code felt stupid since it was outdated the moment you wrote it, and 3 wasn’t really well supported by the community and tooling yet. On the other hand, Python adoption has really taken off since Python 3.5-ish. Python has never been more popular. So while you may wonder what might have been, had the transition been smoother, it’s hard to argue that Python 3 is a failure. All’s well that ends well, I guess? Although it’s sad that Guido felt the need to step down. It’ll be interesting to see where Python goes this decade, now the transition is over and there’s a wealth of possibilities in front of it. I expect there’ll be a lot of people looking to replace JavaScript with Python once you can run it in the browser with WASM.
  • roca 7y ago
    "All's well that ends well" neglects the costs to the community of bad decisions. It also encourages people to think that those decisions must not have been very bad, and not learn from those mistakes. You can see that at work in the responses here. "And I still feel it was the right move. Somehow Python is even more relevant today than it was when this painful process began." I.e. success is thought to justify every decision made along the way. I see this fallacy at work in Linux too. "Linux is successful, therefore haphazard CI and using email to track bugs and patches must be a fine way to operate".
    • mrr54 7y ago
      Haphazard CI and using email to track bugs and patches is a fine way to operate.
• raymondh 7y ago
  This criticism of the dev team seems naïve, "It should not have taken 11 years to get to where we are today." Core developers can make tooling available, but they can't control adoption. That is a user decision. Users switch-over on a timetable governed by their own individual cost-benefit analysis.
  • yjftsjthsd-h 7y ago
    > Core developers can make tooling available, but they can't control adoption Core developers made the design decisions that made nobody want to adopt it. > the ecosystem of users and projects are collectively much better-off than if the transition had not occurred at all. The question seems more like, "could the same benefits have been had with less pain", and a reasonable reading is that the answer is yes. (ex. 4 years of not being able to work with bytes reasonably even if you did need them)
• zmmmmm 7y ago
  It mostly makes me question the wisdom of implementing such a tool in Python in the first place - if you want low level access to raw underlying representations etc, using a super high level scripting language seems like a "wrong tool for the job" scenario. I am sure on the other hand they got a lot of productivity benefits from doing that, which is great, but having taken that tradeoff I don't think it is fair to "sour" on a language when you clearly applied it out of it's domain and then encountered problems due to that.
  • Tanooki_Mario 7y ago
    You can't really say this after the language has supported the feature as a design decision for 15 years and then removes it. Part of the popularity of python is it made things easier for systems programmers. Unless you want us to write everything in C/C++ again?
• adontz 7y ago
  I have never seen such rejection in Django community, despite real problems, like with WSGI design, handling I/O and thus working with bytes a lot. Every huge task, like porting from Python 2 to Python 3 or any other huge task is either everybody's task or just a small group's one. And since latter seems more reasonable to not interfere with ongoing development, former is the only way I have seen such tasks to succeed. Artificial rules to create comfort for one group at the expense of another group, like the following >> This ground rule meant that a mass insertion of b'' prefixes everywhere was not desirable, as that would require developers to think about whether a type was a bytes or str, a distinction they didn't have to worry about on Python 2 because we practically never used the Unicode-based string type in Mercurial. sound pretty much wrong to me. If there is a pain, it should become everybody's pain, or otherwise people will simply burn out and hate own work, like the author did. There is no way porting to Python 3 can be harder than porting to Rust. Rust is statically typed and not garbage collected. Everyone would have to think if they need string or array of bytes anyway, but also, who owns them. Overall, described situation looks like management issue and not a technical one to me. Edit: typos.
  • reubenmorais 7y ago
    > There is no way porting to Python 3 can be harder than porting to Rust. Rust is statically typed and not garbage collected. Everyone would have to think if they need string or array of bytes anyway, but also, who owns them. The author addresses this. The difference is that when porting to Rust you'd likely get a faster and more correct program in the end. (Huge caveat of big rewrites, of course). Whereas with Python 3 they feel like they did all the porting work and got nothing valuable in return.
  • roca 7y ago
    > There is no way porting to Python 3 can be harder than porting to Rust. Rust is statically typed and not garbage collected. Everyone would have to think if they need string or array of bytes anyway, but also, who owns them. The Rust compiler statically checks those decisions, while in Python issues with string types will only be caught at run-time, so everywhere your test suite has missing coverage, porting is likely to introduce regressions. That is one way in which a Rust port would be easier.
• jmilloy 7y ago
  > in Mercurial's code base, most of our string types are binary by design: use of a Unicode based str for representing data is flat out wrong for our use case. I feel like this is the essence of the article: specific constraints/choices of Mecurial made their port to Python 3 difficult. Working with early Python 3 certainly did not help. But there seems to have been some stubbornness here mixed with a lot of retroactive justification. > One was that the added b characters would cause a lot of lines to grow beyond our length limits and we'd have to reformat code. This is almost ridiculous. You are going to write a JIT partial 2to3 instead of just increasing your length limits and/or using an autoformatter? (Of course, it turns out they eventually did do that... after a bit more stubborness regarding the autoformatter.) > So I'm not sure six would have saved enough effort to justify the baggage of integrating a 3rd party package into Mercurial. Couldn't this have been a very occasional copy and paste, instead of a downstream dependency? [six](https://six.readthedocs.io/ https://six.readthedocs.io/) "consists of only one Python file, so it is painless to copy into a project." > Initially, Python 3 had a rather cavalier attitude towards backwards and forwards compatibility. Yes, can't disagree. Early adopters who attempted to write 2- and 3- compatible code suffered the most.
• TeeWEE 7y ago
  We just migrated a big codebase to python 3 in about a year. It was not easy, but also not super hard with tools like futurize, and mypy. Gladly we already had mypy hints, this helped us find a lot of mistakes when (not) using bytes. Now we're on python 3 we're auto-migrate the type hints to be inlined with tools, like com2ann https://github.com/ilevkivskyi/com2ann https://github.com/ilevkivskyi/com2ann And we're auto rewriting code to be more python 3 like with libcst and custom codemods...
• souprock 7y ago
  Not what you want to hear about a version control system: "Python is a dynamic language and there are tons of invariants that aren't caught at compile time and can only be discovered at run time. These invariants cannot all be detected by tests, no matter how good your test coverage is. This is a feature/limitation of dynamic languages. Our users will likely be finding a long tail of miscellaneous bugs on Python 3 for years."
  • indygreg2 7y ago
    C/C++ is a language with limited facilities to ensure correctness at compile time. The languages are riddled with undefined behavior in common features that programmers with multiple decades of experience still get tripped up by. NULL access - the so called "billion dollar mistake" - out of bounds reads and writes, and use after free create a litany of security issues and create massive liability for companies who choose to author software in these languages. Not what you want to hear about an operating system :p
    • otabdeveloper4 7y ago
      "C/C++" is not a language. Good lord, how much ignorance can hackernews handle??
• mbar84 7y ago
  I guess this is as good a time as any to pimp my work in this area: https://pypi.org/project/lib3to6/ https://pypi.org/project/lib3to6/ lib3to6 is a Python compatibility library similar to to BableJS. It translates (most) valid Python 3.7 syntax to valid Python 2.7 and Python 3 syntax (aka. universal python). If you would like to develop with a modern python version and yet still maintain backward compatibility or if you want to bring a legacy codebase forward step by step (my use case), then please have a look.
• epage 7y ago
  I can't believe their leadership de-prioritized the port until the last minute when they have an ecosystem on top of them that also has to port. I feel that was irresponsible. The project lead said to not push `b""` on people. That was a mistake imo that led them down a very frustrating rabbit hole (transformers, `pycompat`) that probably greatly extended their port time. One reason given is to not confuse devs with those details but they are critical details and ones you can't avoid with Rust. This inconsistency makes me wonder if the post is mostly misdirected frustration. A lot of it centers on. I agree about the early python3 releases making it harder. I don't remember what the python leadership's intent was but i think I actually agree with what they did, now. Over my career, I've come to appreciate starting with the ideal and working backwards. This let's you learn what is needed rather than wasting time on speculation (planning or dev) or making a more crippled product. I can understand frustrations with bugs / differences in python versions. I ran into that a lot just within `2.7.*` In my mind, the most notable complaint is the stdlib's mixed efforts in supporting str or bytes. I feel "batteries included" maked this harder. They had to port a lot. Not everything can get the same level of scrutiny, especially from domain experts that represent a variety of use cases. They also can't break compat. If they weren't battries included, the porting efforts would be more directed, pull in the right people, and you can fix things later if you get it wrong. What I find interesting is how different our experiences are that lead to the same place. My frustrations with python are rooted in build tools and packaging and have been loving Rust. EDIT: I'm also surprised at the hostility towards distribution packagers. Instead of working with them to find mutually valid solutions, the express frustration at distributions and cripple themselves in not allowing third-party dependencies.
  • Conan_Kudo 7y ago
    > EDIT: I'm also surprised at the hostility towards distribution packagers. Instead of working with them to find mutually valid solutions, the express frustration at distributions and cripple themselves in not allowing third-party dependencies. These days, it's "cool" to hate your downstreams (y'know, bite the hands that feed you and all that). Seriously though, as one of those "distribution packagers" (Fedora, Mageia, OpenMandriva, and openSUSE!), it sucks that I encounter this more and more often. I try to be somewhat involved in the projects I package and contribute where I can, be it code, advice, or anything in between. Ten years ago, people were generally friendly to me. These days? It's rare to get a thank-you. Usually I get grumbles and anger for daring to ship it in a distro package. I've even had a couple of patches rejected that fix real bugs simply because they were discovered as part of my packaging and testing something because it doesn't happen on the dev's machine in his virtualenv on his Mac...
• bschwindHN 7y ago
  I wonder how many human lives' worth of work has been wasted from the decision to use Python and having to deal with 2/3 transitions, and if it was worth it for the speedup of using an interpreted language.
• gfxgirl 7y ago
  I wish more devs cared about backward compatibility not just in python but in general. I know a particular, very popular library, 60k stars on github, who's maintainers break stuff every month. They don't care how many developers time it wastes they just decide FooBar should really be named FooB and rename it. No Effs given how many people it disrupts. You'd think people would complain but cult of personality and/or popularity of library turns people in to fan boys where they seem to think "If these geniuses are doing it this way then it must be good". .... sigh
• 2T1Qka0rEiPr 7y ago
  Not knowing really anything about Mercurial, the `skip-blame` feature seems interesting, but seems Git doesn't have something similar (has to be constrained when calling `blame`)
• Ohn0 7y ago
  For such a thorough article, I wish there were mention of Python 4
• luord 7y ago
  1. Introduce a new version with the plan of discontinuing the previous version 11 years later (that's almost half of the time that, by then, python had been a thing), that itself was released only three years after the very tool you're talking about was released. 2. Don't even pretend to be interested in trying to do a migration until seven years later. 3. Make sure that your migration plan includes a development cycle that's deliberately hostile to the migration process. 4. ? 5. How could the python maintainers do this to us. The description of the migration process was a good read. The fud afterwards... wasn't. And there were a few inaccuracies (I'm being charitable, some of them were straight up lies). > Python 3.0 was released on December 3, 2008. And it took the better part of a decade for the community to embrace it. False, I've been using python 3, python 3 exclusively, since 2014, for all my projects. > Yes, Python is still healthy today and Python 3 is (finally) being adopted at scale False, same as above. > I am ecstatic the community is finally rallying around Python 3 Again, false. Not only did "the community" rallied around python 3 years ago, he isn't really happy about it, but I'll get to that later. > For nearly 4 years, Python 3 took away the consistent syntax for denoting bytes/Unicode string literals. Or, to put it another way, python 3 was compatible with python 2's string types almost eight years before python 2 reached end of life. > An ecosystem that falters for that long is generally not healthy This entire paragraph was a hypothetical. It seems he really wanted to criticize something that did not happen. > The only language I've seen properly implement higher-order abstractions on top of operating system facilities is Rust And here's where his true point becomes evident: this is a hype piece for a language he found that he likes better. He's just attacking something in his previous language that he thinks is valid just as an attempt to highlight why the new toy is truly better. In short: He felt like complaining about the migration would be a good way to proselytize. Just in case: no, it isn't better, and I say this as someone who currently isn't using python nor rust. I'm using a language that I'm quickly growing to hate more than I do either of them at their worst (no, it's not JavaScript). > if Rust were at its current state 5 years ago, Mercurial would have likely ported from Python 2 to Rust instead of Python 3. As crazy as it initially sounded, I think I agree with that assessment. So... The best he can say about rust is that it might be better than python 3 five years ago that, by his own opinion on everything he wrote before this, was terrible? Well, that's a recommendation not to use rust if I ever saw any. When a hype piece defeats its own point. > And speaking as a maintainer, I have mad respect for the people leading such a large community. No, he doesn't; he used several appeals to emotion beforehand to try to paint them as terrible people. > It should not have taken 11 years to get to where we are today. This statement by itself is a truism that doesn't really mean anything, but the implication is that python 3 is only worthwhile 11 years later and it took that long for it to be so I'll reply to that. No, it didn't. It didn't even take that long for mercurial, they started the migration four and half years ago, not eleven. > am confident it will grow stronger by taking the time to do so What is it to him? He should just move on to rust and be happy with it (sure, there are many people unhappy with it, but he wouldn't take the effort to proselytize if he wasn't). In conclusion, I just don't understand the need to tear something else down to prop up a new thing. I'm sure I would have liked a post about things he could do with rust, but now...
• rurban 7y ago
  For me this is the most exciting outcome: > The only Python 3 feature that Mercurial developers seem to almost universally get excited about is type annotations. We already have some people playing around with pytype using comment-based annotations and pytype has already caught a few bugs. We're eager to go all in on type annotations and uncover lots of dynamic typing bugs and poorly implemented APIs. Over in perl land people still spill their hate on types, which caused hard forks.