5 ms·
> One of the biggest early hurdles in our porting effort was how to overcome the string literals type mismatch between Python 2 and 3. In Python 2, a '' string
by weberc2 7y ago
> One of the biggest early hurdles in our porting effort was how to overcome the string literals type mismatch between Python 2 and 3. In Python 2, a '' string literal is a sequence of bytes. In Python 3, a '' string literal is a sequence of Unicode code points. These are fundamentally different types. And in Mercurial's code base, most of our string types are binary by design: use of a Unicode based str for representing data is flat out wrong for our use case. We knew that Mercurial would need to eventually switch many string literals from '' to b'' to preserve type compatibility. But doing so would be problematic.
> In the early days of Mercurial's Python 3 port in 2015, Mercurial's project maintainer (Matt Mackall) set a ground rule that the Python 3 port shouldn't overly disrupt others: he wanted the Python 3 port to more or less happen in the background and not require every developer to be aware of Python 3's low-level behavior in order to get work done on the existing Python 2 code base. This may seem like a questionable decision (and I probably disagreed with him to some extent at the time because I was doing Python 3 porting work and the decision constrained this work). But it was the correct decision. Matt knew that it would be years before the Python 3 port was either necessary or resulted in a meaningful return on investment (the value proposition of Python 3 has always been weak to Mercurial because Python 3 doesn't demonstrate a compelling advantage over Python 2 for our use case).
As a general rule, this seems like good practice, but surely b-strings, print_function, etc are a trivial upfront cost, and one that would have to be paid sooner or later anyway?
- kingemer 7y agoIt sounds like the cost was non trivial for them, partially because they weren’t allowed to break things for python2, or even disrupt the efforts of those using it. The language wasn’t ready for the transition, but it feels like it may have been even harder on them because of the requirements imposed on their project.
- kingemer 7y agoConsidering how much opposition there is in moving to python3, has there been any significant effort in the community keeping python2 alive?
- acdha 7y agoMost of what opposition there is comes from people with projects which are some combination of under-staffed, poorly tested, or with a marginal approach to scheduling necessary maintenance work. That is not a great place to expect to find reliable maintenance contributions. Where you are more likely to find this is from the major Linux vendors: e.g. Red Hat is committed to support it through 2024 and I would expect that they won't be alone in offering paid support for remaining users.
- weberc2 7y agoI'm actually (pleasantly) surprised that someone like Google (or a league of someones like Google) who have deep pockets and so much Python 2 that it's cheaper to maintain Python 2 than it is to port to Python 3. Perhaps they (correctly) were concerned about the ecosystem moving on toward Python 3, leaving them behind?
- weberc2 7y agoRight, they could have paid a nontrivial cost (asking devs to use b-strings, print_function, etc) but didn't by fiat, choosing instead to pay a greater cost during the migration in addition to the nontrivial cost. My comment expresses skepticism about that decision.
- novok 7y agoHaving done a py2 to py3 migration and ran into the same issues, these wouldn't of been that big of a deal IF python had static typing from the get go. The static compiler would notice all the breaking type changes at compile time and you can systematically fix all of them at once. You wouldn't miss one or two and have to run your unit testing suite to exercise the type system underneath. I really believe big breaking changes like this in a language causing migration stagnation is a property of dynamically typed languages. With other statically typed languages like swift or rust, it happened quite frequently but wasn't as big of a deal in practice.
- TylerE 7y agoPython with static typing wouldn't be python. It wouldn't even be python-adjacent.
- lacker 7y agoIt seems like a lot of the cost was when system libraries made different choices than Mercurial would have made. For example, the Python 3 filesystem libraries often used unicode as a wrapper around an underlying bytes interface, and Mercurial really wanted to be able to pass bytes directly to the underlying interface. So it isn't just that they had to update their data types, they also had to adjust code to work with system libraries of slightly different semantics.
- wnoise 7y agoThe python interface usually would take bytes, and if it did would also return bytes. But there were a lot of things that didn't take arguments, so always returned Unicode strings. Instead of e.g. getcwd(), you would have to use getcwdb(). Which naturally didn't have an equivalent in python 2 though they did add the complementary getcwdu() (which one should basically never use).