7 ms·

> The minor improvements of Python 3 did not warrant breaking backwards compatibility and most could have been handled in a way that would not break it opt-in d

by orf 6y ago

> The minor improvements of Python 3 did not warrant breaking backwards compatibility and most could have been handled in a way that would not break it opt-in directives. There where large changes to fundamental parts of the type system, as well as core types. Pretending this isn’t the case belays ignorance or at the very least cherry picking. How would you have handled the string/bytes split in a way that’s backwards compatible? Or the removal of old-style classes?

ktpsns 6y ago
I think some modular approach could have solved the incompatiblity issue, such as "from future import ...". Shorthands could have been invented to define everything in a single line. Perl5 has similar flags ("use strict"), and Racket brings it even further to define the whole fucking language of the rest of the file ("#lang racket/gui"). Having the language being choosable by the user is against the "zen of python", I guess. In other words: Such an attemp does not feel "pythonic".
- orf 6y ago
  For some syntactical sugar, like print-function, sure. But there are more fundamental changes that couldn’t be papered over.
  - jamiesonbecker 6y ago
    Fundamental changes shouldn't have been papered over by calling the new language Python, as it was a fundamentally different language at that point.
    - orf 6y ago
      No, it’s the same language but with different semantics around a specific type. That’s not a different language and code can co-exist with a bit of thought.
    - JasonFruit 6y ago
      Every language goes through this at some point in its development: flaws that limit future development have to be fixed. Should every language rename itself and split its community at that point? That seems like an extreme response to a common problem.
      - Chris2048 6y ago
        > Should every language rename itself and split its community at that point? yes. If breaking, fundamental changes are common, that's a problem.
        JasonFruit 6y ago
        That people can make an initial plan that is self-consistent, logical, and foresees and provides for all future use-cases is a basic tenet of waterfall-style development. The history of software engineering does not uphold that principle. Why would it be different for language designers?
        Chris2048 6y ago
        Just because an unforeseen issue arises, doesn't mean you need to introduce a breaking change right away.
    - djur 6y ago
      The "new language" is called Python 3.
      - a1369209993 6y ago
        Yes, yes it is. And, like "Perl 6" and to a lesser extent "C++", that name is misleading (and therefore bad), because there is already a different language called "Python" (respectively "Perl", "C"), with significant superficial similarities that it could be confused with.
        lizmat 6y ago
        Please note that the misleading part of Perl 6 has been fixed by renaming it to the Raku Programming Language (https://raku.org https://raku.org using the #rakulang tag on social media).
nicoburns 6y ago
> How would you have handled the string/bytes split in a way that’s backwards compatible? My understanding is that the corresponding types are available in both 2 and 3, they're just named differently. A different one is "string". So you could have had some kind of mode directive at the top of the file which controlled which version that file was in, and allow files from 2 and 3 to be run together.
- orf 6y ago
  Actually think about it. bytes is str in Python 2. There is no bytes type in py2. How would a per-file directive (of all things) help? What if one function running in “py2 mode” returned a string-that-is-actually-bytes, how would a function in “py3 mode” consume it? What would the type be? If different, how would it be detected or converted? What if it retuned a utf8 string OR bytes? What if that py3 function then passed it to a py2 function - would it become a string again? Would you have two string types - py2string that accepts anything and py3string that only works with utf8? How would this all work with C modules?
  - nicoburns 6y ago
    > What if one function running in “py2 mode” returned a string-that-is-actually-bytes, how would a function in “py3 mode” consume it? What would the type be? It would be bytes. Because py2 string === py3 bytes. > What if that py3 function then passed it to a py2 function - would it become a string again? Yes > Would you have two string types - py2string that accepts anything and py3string that only works with utf8? Yes. You already have those two types in python3. bytes and string. You'd just alias those as string and utf8 or whatever you want to call it in python2. How would this all work with C modules? They'd have specify which mode they were working with too.
    - orf 6y ago
      But all this would require huge rewrites of code and would never be backward compatible. You’re trading “py2 vs py3” with “py2 mode vs py3 mode”. So you’d have some magic code that switches py2str to bytes. Which means every py3 caller has to cast bytes into a string to do anything useful with it, because returning strings is the most common case. Then that code has to be removed when the code it’s calling is updated to py3 mode. Which is basically the blue/green issue you see with async functions but way, way worse. Then you’d need to handle subclasses, wrappers of bytes/str, returning collections of strings across py2/py3 boundaries (would these be copies? Different types? How would type(value[0]) work?), ending up with mixed lists/dicts of bytes and strings depending on the function context, etc etc. It would become an absolute complete clusterfuck of corner cases that would have killed the language outright.
Blikkentrekker 6y ago
> How would you have handled the string/bytes split in a way that’s backwards compatible? A language pragma. All functions that return `bytes` continue to do so unless specifically opted in on a per file basis, then they return `unicode`. `str` thus returns `bytes` as it does in 2, unless the pragma ask otherwise. > Or the removal of old-style classes? They would obviously not be removed and still be available but depræcated.
- orf 6y ago
  > All functions that return `bytes` continue to do so unless specifically opted in on a per file basis, then they return `unicode`. Nothing in py2 returns bytes. They all return strings. That is the issue. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > They would obviously not be removed and still be available but depræcated. Having two almost separate object models in the same language is rather silly.
  - Blikkentrekker 6y ago
    > Nothing in py2 returns bytes. They all return strings. That is the issue. No, that is not an issue, that is semantics. What one calls it does not change the behavior. And aside that the system could perfectly well be designed that this pragma changes that `str` is synonymous with either `bytes` or `unicode` depending on it's state. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? You would now which is which by using the pragma or not. Not using the pragma defaults to the old behavior, as said, one only receives the new, breaking behavior, when one opts in. Python could even support always opting in by a configuration file option for those that really want it and don't want to add the pragma at the top of every file. > A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. Opposed to the burden they already had of maintaining a 2 and 3 version? Any new code can of course always return `unicode` rather than `str` which in this scheme is normally `bytes` but becomes `unicode` with the pragma. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > Having two almost separate object models in the same language is rather silly. Yes, it is, and you will find that most languages are full of such legacy things that no new code uses but are simply for legacy purposes. “It is silly.” turns out to be a rather small price to pay to achieve. “We have not broken backwards compatibility.”.
naniwaduni 6y ago
Let's not pretend that py3's string changes weren't fundamentally wrong and didn't create years of issues trying to decode things that could properly be arbitrary byte sacks as utf-8. So my answer is that it was a deeply misconceived change that shouldn't have been made at all, let alone been taken as the cornerstone of a "necessary" break in backward compatibility.
- orf 6y ago
  The string changes where both necessary and correct. There is a difference between bytes and strings and treating them as the same led to so many issues. Thank god I’ve not seen a UnicodeDecodeError in decades. And the ecosystem agrees.
- nerdponx 6y ago
  what's wrong about strings representing text? You're not making an argument about backward compatibility here, you're making a strong claim that representing text as a sequence of Unicode code points is fundamentally wrong. I have never heard anyone make this point before, and I am inclined to disagree, but I'm curious what your reasoning is for it.
  - naniwaduni 6y ago
    Indeed, representing text as a sequence of Unicode code points is fundamentally wrong. There are no operations on sequences of Unicode code points that are more correct than an analogous operation on bytes. (Everyone's favourite example, length, actually becomes less correct—a byte array's length at least corresponds to the amount of space one might have to allocate for it in a particular encoding. A length in codepoints is absolutely meaningless both technically and linguistically. And this is, for what little it's worth, close to the only operation you can do on a string without imposing additional restrictions about its context.)
    - orf 6y ago
      > There are no operations on sequences of Unicode code points that are more correct than an analogous operation on bytes. That’s ridiculous. Uppercasing/lowercasing, slicing, “startswith”, splitting, etc etc. Your statement is correct if you only care about ascii.
imtringued 6y ago
When I was learning Python 6 years ago I was the only one using Python 3 in my group because I use arch linux. It was very basic code and everyone basically solved the same problem. Everyone else's code didn't work on my machine because print is not a statement in Python 3. That's just plain stupid. Just print a warning and add a python2 flag that hides the warning. Don't release a major version because of something trivial like this.
- orf 6y ago
  They didn’t release A major version because they changed print from a statement to a function.
- TheGoddessInari 6y ago
  Python gave everyone 12 years to deal with version 3 being the way forward. There are many fundamental changes. The fact that people seem to complain exclusively after Python 2' end of life a year ago feels a little telling. Perl's community roffle stomped their previous vision for Perl 6. Python community wasn't vocal about this being a bad change. Rather the opposite, very loud support. Keep in mind, I dislike Python either way, but I'm not one of the devs that complains about continuing education requirements, or language adding things over each 10 year period. I can work in Python just fine, but that doesn't mean it feels nice & hygienic to use for me personally.
musicale 6y ago
> How would you have handled the string/bytes split in a way that’s backwards compatible? Or the removal of old-style classes? I'm not sure it's the best way to handle it, but I would have been fine with: from __python2__ import * for full backward compatibility; or, more explicitly: from __python2__ import ascii_strings, old_style_classes, print_statement, ... As the parent poster mentions, several other popular languages and systems (C++, Java, etc.) have done a pretty decent job preserving backward compatibility, for good reason: it saves millions of hours of human effort. It's embarrassing and disappointing that Python simply blew it with the Python 2 to 3 transition. Maybe we could still evolve pypi to support a compatibility layer to allow easy mixing of python2 and python3 code, but I get the feeling that Python 3 has poisoned the well.

by orf 6y ago

ktpsns 6y ago

I think some modular approach could have solved the incompatiblity issue, such as "from future import ...". Shorthands could have been invented to define everything in a single line. Perl5 has similar flags ("use strict"), and Racket brings it even further to define the whole fucking language of the rest of the file ("#lang racket/gui"). Having the language being choosable by the user is against the "zen of python", I guess. In other words: Such an attemp does not feel "pythonic".

orf 6y ago
For some syntactical sugar, like print-function, sure. But there are more fundamental changes that couldn’t be papered over.
- jamiesonbecker 6y ago
  Fundamental changes shouldn't have been papered over by calling the new language Python, as it was a fundamentally different language at that point.
  - orf 6y ago
    No, it’s the same language but with different semantics around a specific type. That’s not a different language and code can co-exist with a bit of thought.
  - JasonFruit 6y ago
    Every language goes through this at some point in its development: flaws that limit future development have to be fixed. Should every language rename itself and split its community at that point? That seems like an extreme response to a common problem.
    - Chris2048 6y ago
      > Should every language rename itself and split its community at that point? yes. If breaking, fundamental changes are common, that's a problem.
      - JasonFruit 6y ago
        That people can make an initial plan that is self-consistent, logical, and foresees and provides for all future use-cases is a basic tenet of waterfall-style development. The history of software engineering does not uphold that principle. Why would it be different for language designers?
        Chris2048 6y ago
        Just because an unforeseen issue arises, doesn't mean you need to introduce a breaking change right away.
  - djur 6y ago
    The "new language" is called Python 3.
    - a1369209993 6y ago
      Yes, yes it is. And, like "Perl 6" and to a lesser extent "C++", that name is misleading (and therefore bad), because there is already a different language called "Python" (respectively "Perl", "C"), with significant superficial similarities that it could be confused with.
      - lizmat 6y ago
        Please note that the misleading part of Perl 6 has been fixed by renaming it to the Raku Programming Language (https://raku.org https://raku.org using the #rakulang tag on social media).

nicoburns 6y ago

> How would you have handled the string/bytes split in a way that’s backwards compatible? My understanding is that the corresponding types are available in both 2 and 3, they're just named differently. A different one is "string". So you could have had some kind of mode directive at the top of the file which controlled which version that file was in, and allow files from 2 and 3 to be run together.

orf 6y ago
Actually think about it. bytes is str in Python 2. There is no bytes type in py2. How would a per-file directive (of all things) help? What if one function running in “py2 mode” returned a string-that-is-actually-bytes, how would a function in “py3 mode” consume it? What would the type be? If different, how would it be detected or converted? What if it retuned a utf8 string OR bytes? What if that py3 function then passed it to a py2 function - would it become a string again? Would you have two string types - py2string that accepts anything and py3string that only works with utf8? How would this all work with C modules?
- nicoburns 6y ago
  > What if one function running in “py2 mode” returned a string-that-is-actually-bytes, how would a function in “py3 mode” consume it? What would the type be? It would be bytes. Because py2 string === py3 bytes. > What if that py3 function then passed it to a py2 function - would it become a string again? Yes > Would you have two string types - py2string that accepts anything and py3string that only works with utf8? Yes. You already have those two types in python3. bytes and string. You'd just alias those as string and utf8 or whatever you want to call it in python2. How would this all work with C modules? They'd have specify which mode they were working with too.
  - orf 6y ago
    But all this would require huge rewrites of code and would never be backward compatible. You’re trading “py2 vs py3” with “py2 mode vs py3 mode”. So you’d have some magic code that switches py2str to bytes. Which means every py3 caller has to cast bytes into a string to do anything useful with it, because returning strings is the most common case. Then that code has to be removed when the code it’s calling is updated to py3 mode. Which is basically the blue/green issue you see with async functions but way, way worse. Then you’d need to handle subclasses, wrappers of bytes/str, returning collections of strings across py2/py3 boundaries (would these be copies? Different types? How would type(value[0]) work?), ending up with mixed lists/dicts of bytes and strings depending on the function context, etc etc. It would become an absolute complete clusterfuck of corner cases that would have killed the language outright.

Blikkentrekker 6y ago

> How would you have handled the string/bytes split in a way that’s backwards compatible? A language pragma. All functions that return `bytes` continue to do so unless specifically opted in on a per file basis, then they return `unicode`. `str` thus returns `bytes` as it does in 2, unless the pragma ask otherwise. > Or the removal of old-style classes? They would obviously not be removed and still be available but depræcated.

orf 6y ago
> All functions that return `bytes` continue to do so unless specifically opted in on a per file basis, then they return `unicode`. Nothing in py2 returns bytes. They all return strings. That is the issue. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > They would obviously not be removed and still be available but depræcated. Having two almost separate object models in the same language is rather silly.
- Blikkentrekker 6y ago
  > Nothing in py2 returns bytes. They all return strings. That is the issue. No, that is not an issue, that is semantics. What one calls it does not change the behavior. And aside that the system could perfectly well be designed that this pragma changes that `str` is synonymous with either `bytes` or `unicode` depending on it's state. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? You would now which is which by using the pragma or not. Not using the pragma defaults to the old behavior, as said, one only receives the new, breaking behavior, when one opts in. Python could even support always opting in by a configuration file option for those that really want it and don't want to add the pragma at the top of every file. > A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. Opposed to the burden they already had of maintaining a 2 and 3 version? Any new code can of course always return `unicode` rather than `str` which in this scheme is normally `bytes` but becomes `unicode` with the pragma. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > Having two almost separate object models in the same language is rather silly. Yes, it is, and you will find that most languages are full of such legacy things that no new code uses but are simply for legacy purposes. “It is silly.” turns out to be a rather small price to pay to achieve. “We have not broken backwards compatibility.”.

naniwaduni 6y ago

Let's not pretend that py3's string changes weren't fundamentally wrong and didn't create years of issues trying to decode things that could properly be arbitrary byte sacks as utf-8. So my answer is that it was a deeply misconceived change that shouldn't have been made at all, let alone been taken as the cornerstone of a "necessary" break in backward compatibility.

orf 6y ago
The string changes where both necessary and correct. There is a difference between bytes and strings and treating them as the same led to so many issues. Thank god I’ve not seen a UnicodeDecodeError in decades. And the ecosystem agrees.
nerdponx 6y ago
what's wrong about strings representing text? You're not making an argument about backward compatibility here, you're making a strong claim that representing text as a sequence of Unicode code points is fundamentally wrong. I have never heard anyone make this point before, and I am inclined to disagree, but I'm curious what your reasoning is for it.
- naniwaduni 6y ago
  Indeed, representing text as a sequence of Unicode code points is fundamentally wrong. There are no operations on sequences of Unicode code points that are more correct than an analogous operation on bytes. (Everyone's favourite example, length, actually becomes less correct—a byte array's length at least corresponds to the amount of space one might have to allocate for it in a particular encoding. A length in codepoints is absolutely meaningless both technically and linguistically. And this is, for what little it's worth, close to the only operation you can do on a string without imposing additional restrictions about its context.)
  - orf 6y ago
    > There are no operations on sequences of Unicode code points that are more correct than an analogous operation on bytes. That’s ridiculous. Uppercasing/lowercasing, slicing, “startswith”, splitting, etc etc. Your statement is correct if you only care about ascii.

imtringued 6y ago

When I was learning Python 6 years ago I was the only one using Python 3 in my group because I use arch linux. It was very basic code and everyone basically solved the same problem. Everyone else's code didn't work on my machine because print is not a statement in Python 3. That's just plain stupid. Just print a warning and add a python2 flag that hides the warning. Don't release a major version because of something trivial like this.

orf 6y ago
They didn’t release A major version because they changed print from a statement to a function.
TheGoddessInari 6y ago
Python gave everyone 12 years to deal with version 3 being the way forward. There are many fundamental changes. The fact that people seem to complain exclusively after Python 2' end of life a year ago feels a little telling. Perl's community roffle stomped their previous vision for Perl 6. Python community wasn't vocal about this being a bad change. Rather the opposite, very loud support. Keep in mind, I dislike Python either way, but I'm not one of the devs that complains about continuing education requirements, or language adding things over each 10 year period. I can work in Python just fine, but that doesn't mean it feels nice & hygienic to use for me personally.

musicale 6y ago

> How would you have handled the string/bytes split in a way that’s backwards compatible? Or the removal of old-style classes? I'm not sure it's the best way to handle it, but I would have been fine with: from __python2__ import * for full backward compatibility; or, more explicitly: from __python2__ import ascii_strings, old_style_classes, print_statement, ... As the parent poster mentions, several other popular languages and systems (C++, Java, etc.) have done a pretty decent job preserving backward compatibility, for good reason: it saves millions of hours of human effort. It's embarrassing and disappointing that Python simply blew it with the Python 2 to 3 transition. Maybe we could still evolve pypi to support a compatibility layer to allow easy mixing of python2 and python3 code, but I get the feeling that Python 3 has poisoned the well.