13 ms·

I think a lot of important lessons got learned in both cases. Clearly perl6 should have had a different name. But I think python2->python3 could've been much le

I think a lot of important lessons got learned in both cases. Clearly perl6 should have had a different name. But I think python2->python3 could've been much less painful if they'd known to prioritize single-codebase compatibility from the very beginning. I think you can see that lesson applied with e.g. Rust editions, which as far as I can tell have been a complete success.

xmprt 6y ago
Single codebase compatibility meaning that you can have python2 and python3 code in the same application? Isn't that significantly harder with an interpreted language or am I missing something?
- gdwatson 6y ago
  Single-codebase compatibility means that the same code can run under both Python 2 and Python 3. The initial expectation was that translation tools would solve this problem, but it didn’t really work out that way. Adding language features and library shims to make it possible to write pidgin Python that would run under either version meant that you could migrate libraries and parts of large codebases one at a time until the whole thing ran under Python 3.
  - uranusjr 6y ago
    “Pidgin Python” is such an apt term, why am I only seeing it when it is no longer relevant!
  - dwheeler 6y ago
    That's the main working solution I found: code works in both versions. The problem is that it's way trickier than it should be. Has they made that relatively easy, the Python 2->3 transition would have had a much smoother "normal" upgrade process.
- deleted 6y ago
  [deleted]
- slaymaker1907 6y ago
  It's a mostly solved problem with Racket. What they probably should have done was have python 2 code somehow declare itself as python 2 (Racket does this with #lang at the top of files). Then, just have a python 2 compatibility layer that works in two steps. First step is to compile/parse it into a similar form as python 3. Additionally, provide a small python 2 runtime which provides different versions of the functions which changed from 2 to 3. I think the two steps are important because some stuff is easier to solve via compilation like "print" while other stuff may be only possible at runtime like strings being unicode. You would still have some differences which can't be papered over, but it would have made writing code that works in both python 2 and 3 much easier.
Blikkentrekker 6y ago
Hindsight might very well be easy in this case, but I cannot think otherwise than finding the Python developers to have been ridiculously naïve in how they handled this, and foolish to even begin it from the start. The minor improvements of Python 3 did not warrant breaking backwards compatibility and most could have been handled in a way that would not break it opt-in directives. The very swarms of users that chanted “just upgrade” as if that not incur a significant cost also seemed ridiculously naïve to me, not understanding the real cost that large projects have by having to rewrite very extensive codebases and dealing the potential regressions that that might involve. Everything about the switch, from it's very conception, to it's execution, was handled in a veritably disastrous way by the team, that really did not seem to appreciate even a fraction of what obviously is involved with projects that have millions of lines of code and of course would rather not have to rewrite that all. This is why many projects such as Linux, Windows, Rust, Cobol, Fortran, C, and C++ take backwards compatibility quite serious. Serious enterprises do not like to invest in something if it mean that 10 years later they would have to rewrite their entire codebase again. Even on my home computer, I simply do not have the time to rewrite the many Python 2 scripts that have written over the years that run my computer. — it is cumbersome enough that once in a while part of my desktop stops functioning because my distribution removed a Python 2 library which I had relied upon as a system library that I now have to install as a user library but that was hitherto quite easily fixed. Do these men think that time is free?
- brodie 6y ago
  The Python core devs did not have the time or motivation to support the old codepaths in the CPython runtime, and the legacy code was getting in the way of a lot of longtime wants and needs for improving performance, runtime maintainability, language ergonomics, and the standard library. They also specifically increased the major revision number to signal their intent to move on from that legacy. But you kind of addressed your entire spiel: Hindsight is exceedingly easy. They didn't realize how inadequate their migration tooling was, or how very entrenched Python 2 is in various places. It's hard when you don't know what you don't know and you're highly motivated by hopeful aspirations.
  - 6y ago
pjmlp 6y ago
I am still not sold to Rust editions and think in the long run they won't be much different from solutions in other platforms. There will come the day when something will change semantics, or require different kinds of runtime support across editions, and then the headaches how to link binary crates from different editions will start. Editions to me appear only to work, provided everything is compiled from source code with the same compiler, aware of all editions that came into use.
- johncolanduoni 6y ago
  There's no ABI compatibility between different Rust compiler versions as it is, so I don't see how editions will break a compatibility that doesn't exist.
  - pjmlp 6y ago
    Which is my point about it being half solution.
- fluffything 6y ago
  You keep repeating FUD. Read the damn Editions RFC. The community agreed that no semantics or ABi breaking changes will land to Rust, EVER. This is not a lesson from Py th on, but from C++, which introduces breaking changes every single release, which are much smaller than Python but still a pain in million LOc code bases. If that ever happens, it was agreed that the result would be a different language, with a different name. That is, editions don’t have this problem because the FUD that you are trying to spread every single time this issue comes up cannot happen, by design. Your argument “Rust editions don’t solve this problem because they don’t handle semantic or ABI changes” is false, because in Rust there CANNOT be any semantics or ABI changes, and editions handle this situation just fine. In the context of Python 2 vs 3 this argument makes even less sense, because editions allows combining libraries from different editions ina forward and backward compatible way without issues. Source: work on multiple >500kLOC Rust code base and one >1 million LOC and they all use crates from all editions, and mix & match them, without any issues, doing LTO across them, using dynamic libraries, and all possible combinations of binaries across 4 major platforms.
Donckele 6y ago
“Rust editions, which as far as I can tell have been a complete success.” Can anyone tell me why statements such as these get the emotions going?
- richardfey 6y ago
  Hello AI!
  - sgt 6y ago
    Hello AI!
simias 6y ago
>Rust editions, which as far as I can tell have been a complete success. Rust's commitment to backward compatibility is certainly extremely commendable, but I don't think the language went through anything resembling the switch from Python 2 to Python 3 in terms of breakage. Some of the changes in Python 3 are very fundamental. Imagine if Rust had shipped without String/str and they were added after the fact, would Rust manage to avoid splitting the ecosystem? That's an open question as far as I'm concerned. And I also hope that we never find out. Rust's fundamentals have proven to be very solid so far. Having things like OsStrings (something missing from most programming languages, including Python 3 AFAIK) shows a great amount of foresight and understanding of the problem space. Contrast that with Go which seems very intent on completely ignoring 30 years of programming language evolution.
- stjohnswarts 6y ago
  Well they had a lot of languages to look at and say "ummm that didn't work out for them so well, let's not do that" such as the Python fiasco.
linspace 6y ago
I hear very often criticism of Python's transition but considering how insanely popular the language has grown I don't understand why.
- EamonnMR 6y ago
  Python2 was poised to take over the world and be the next Java. 3 is losing ground to Node, Go, Rust, even Lua. 3 is a really fun and productive language to work in as long as you don't need to think about bytes.
  - paulryanrogers 6y ago
    My guess is Python wouldn't have done much better anyway. The GIL and lack of client side support kept it out of a lot of use cases.
    - Demiurge 6y ago
      Really? I’ve been using Python for 15 years and never heard anyone say “GIL”. How and what is GIL preventing?
      - paulryanrogers 6y ago
        https://wiki.python.org/moin/GlobalInterpreterLock https://wiki.python.org/moin/GlobalInterpreterLock
      - stjohnswarts 6y ago
        It causes issues with running low overhead multithreaded code. I get by with the multiprocessing library, but then again I don't have a lot of threads (10 is the most I've ever needed), some people want to run hundreds of "light threads" depending on the type of programming that you are looking to do.
- stjohnswarts 6y ago
  Statistics and probability.

xmprt 6y ago

Single codebase compatibility meaning that you can have python2 and python3 code in the same application? Isn't that significantly harder with an interpreted language or am I missing something?

gdwatson 6y ago
Single-codebase compatibility means that the same code can run under both Python 2 and Python 3. The initial expectation was that translation tools would solve this problem, but it didn’t really work out that way. Adding language features and library shims to make it possible to write pidgin Python that would run under either version meant that you could migrate libraries and parts of large codebases one at a time until the whole thing ran under Python 3.
- uranusjr 6y ago
  “Pidgin Python” is such an apt term, why am I only seeing it when it is no longer relevant!
- dwheeler 6y ago
  That's the main working solution I found: code works in both versions. The problem is that it's way trickier than it should be. Has they made that relatively easy, the Python 2->3 transition would have had a much smoother "normal" upgrade process.
deleted 6y ago
[deleted]
slaymaker1907 6y ago
It's a mostly solved problem with Racket. What they probably should have done was have python 2 code somehow declare itself as python 2 (Racket does this with #lang at the top of files). Then, just have a python 2 compatibility layer that works in two steps. First step is to compile/parse it into a similar form as python 3. Additionally, provide a small python 2 runtime which provides different versions of the functions which changed from 2 to 3. I think the two steps are important because some stuff is easier to solve via compilation like "print" while other stuff may be only possible at runtime like strings being unicode. You would still have some differences which can't be papered over, but it would have made writing code that works in both python 2 and 3 much easier.

Blikkentrekker 6y ago

Hindsight might very well be easy in this case, but I cannot think otherwise than finding the Python developers to have been ridiculously naïve in how they handled this, and foolish to even begin it from the start. The minor improvements of Python 3 did not warrant breaking backwards compatibility and most could have been handled in a way that would not break it opt-in directives. The very swarms of users that chanted “just upgrade” as if that not incur a significant cost also seemed ridiculously naïve to me, not understanding the real cost that large projects have by having to rewrite very extensive codebases and dealing the potential regressions that that might involve. Everything about the switch, from it's very conception, to it's execution, was handled in a veritably disastrous way by the team, that really did not seem to appreciate even a fraction of what obviously is involved with projects that have millions of lines of code and of course would rather not have to rewrite that all. This is why many projects such as Linux, Windows, Rust, Cobol, Fortran, C, and C++ take backwards compatibility quite serious. Serious enterprises do not like to invest in something if it mean that 10 years later they would have to rewrite their entire codebase again. Even on my home computer, I simply do not have the time to rewrite the many Python 2 scripts that have written over the years that run my computer. — it is cumbersome enough that once in a while part of my desktop stops functioning because my distribution removed a Python 2 library which I had relied upon as a system library that I now have to install as a user library but that was hitherto quite easily fixed. Do these men think that time is free?

brodie 6y ago
The Python core devs did not have the time or motivation to support the old codepaths in the CPython runtime, and the legacy code was getting in the way of a lot of longtime wants and needs for improving performance, runtime maintainability, language ergonomics, and the standard library. They also specifically increased the major revision number to signal their intent to move on from that legacy. But you kind of addressed your entire spiel: Hindsight is exceedingly easy. They didn't realize how inadequate their migration tooling was, or how very entrenched Python 2 is in various places. It's hard when you don't know what you don't know and you're highly motivated by hopeful aspirations.
- 6y ago

pjmlp 6y ago

I am still not sold to Rust editions and think in the long run they won't be much different from solutions in other platforms. There will come the day when something will change semantics, or require different kinds of runtime support across editions, and then the headaches how to link binary crates from different editions will start. Editions to me appear only to work, provided everything is compiled from source code with the same compiler, aware of all editions that came into use.

johncolanduoni 6y ago
There's no ABI compatibility between different Rust compiler versions as it is, so I don't see how editions will break a compatibility that doesn't exist.
- pjmlp 6y ago
  Which is my point about it being half solution.
fluffything 6y ago
You keep repeating FUD. Read the damn Editions RFC. The community agreed that no semantics or ABi breaking changes will land to Rust, EVER. This is not a lesson from Py th on, but from C++, which introduces breaking changes every single release, which are much smaller than Python but still a pain in million LOc code bases. If that ever happens, it was agreed that the result would be a different language, with a different name. That is, editions don’t have this problem because the FUD that you are trying to spread every single time this issue comes up cannot happen, by design. Your argument “Rust editions don’t solve this problem because they don’t handle semantic or ABI changes” is false, because in Rust there CANNOT be any semantics or ABI changes, and editions handle this situation just fine. In the context of Python 2 vs 3 this argument makes even less sense, because editions allows combining libraries from different editions ina forward and backward compatible way without issues. Source: work on multiple >500kLOC Rust code base and one >1 million LOC and they all use crates from all editions, and mix & match them, without any issues, doing LTO across them, using dynamic libraries, and all possible combinations of binaries across 4 major platforms.

Donckele 6y ago

“Rust editions, which as far as I can tell have been a complete success.” Can anyone tell me why statements such as these get the emotions going?

richardfey 6y ago
Hello AI!
- sgt 6y ago
  Hello AI!

simias 6y ago

>Rust editions, which as far as I can tell have been a complete success. Rust's commitment to backward compatibility is certainly extremely commendable, but I don't think the language went through anything resembling the switch from Python 2 to Python 3 in terms of breakage. Some of the changes in Python 3 are very fundamental. Imagine if Rust had shipped without String/str and they were added after the fact, would Rust manage to avoid splitting the ecosystem? That's an open question as far as I'm concerned. And I also hope that we never find out. Rust's fundamentals have proven to be very solid so far. Having things like OsStrings (something missing from most programming languages, including Python 3 AFAIK) shows a great amount of foresight and understanding of the problem space. Contrast that with Go which seems very intent on completely ignoring 30 years of programming language evolution.

stjohnswarts 6y ago
Well they had a lot of languages to look at and say "ummm that didn't work out for them so well, let's not do that" such as the Python fiasco.

linspace 6y ago

I hear very often criticism of Python's transition but considering how insanely popular the language has grown I don't understand why.

EamonnMR 6y ago
Python2 was poised to take over the world and be the next Java. 3 is losing ground to Node, Go, Rust, even Lua. 3 is a really fun and productive language to work in as long as you don't need to think about bytes.
- paulryanrogers 6y ago
  My guess is Python wouldn't have done much better anyway. The GIL and lack of client side support kept it out of a lot of use cases.
  - Demiurge 6y ago
    Really? I’ve been using Python for 15 years and never heard anyone say “GIL”. How and what is GIL preventing?
    - paulryanrogers 6y ago
      https://wiki.python.org/moin/GlobalInterpreterLock https://wiki.python.org/moin/GlobalInterpreterLock
    - stjohnswarts 6y ago
      It causes issues with running low overhead multithreaded code. I get by with the multiprocessing library, but then again I don't have a lot of threads (10 is the most I've ever needed), some people want to run hundreds of "light threads" depending on the type of programming that you are looking to do.
stjohnswarts 6y ago
Statistics and probability.