8 ms·

> How would you have handled the string/bytes split in a way that’s backwards compatible? A language pragma. All functions that return `bytes` continue to do

> How would you have handled the string/bytes split in a way that’s backwards compatible? A language pragma. All functions that return `bytes` continue to do so unless specifically opted in on a per file basis, then they return `unicode`. `str` thus returns `bytes` as it does in 2, unless the pragma ask otherwise. > Or the removal of old-style classes? They would obviously not be removed and still be available but depræcated.

orf 6y ago
> All functions that return `bytes` continue to do so unless specifically opted in on a per file basis, then they return `unicode`. Nothing in py2 returns bytes. They all return strings. That is the issue. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > They would obviously not be removed and still be available but depræcated. Having two almost separate object models in the same language is rather silly.
- Blikkentrekker 6y ago
  > Nothing in py2 returns bytes. They all return strings. That is the issue. No, that is not an issue, that is semantics. What one calls it does not change the behavior. And aside that the system could perfectly well be designed that this pragma changes that `str` is synonymous with either `bytes` or `unicode` depending on it's state. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? You would now which is which by using the pragma or not. Not using the pragma defaults to the old behavior, as said, one only receives the new, breaking behavior, when one opts in. Python could even support always opting in by a configuration file option for those that really want it and don't want to add the pragma at the top of every file. > A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. Opposed to the burden they already had of maintaining a 2 and 3 version? Any new code can of course always return `unicode` rather than `str` which in this scheme is normally `bytes` but becomes `unicode` with the pragma. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > Having two almost separate object models in the same language is rather silly. Yes, it is, and you will find that most languages are full of such legacy things that no new code uses but are simply for legacy purposes. “It is silly.” turns out to be a rather small price to pay to achieve. “We have not broken backwards compatibility.”.
  - orf 6y ago
    I don’t really have the time of inclination to continue arguing, but I will point out that you say all this as though the approach the team took failed. It worked. The ecosystem is on py3. You can imagine some world with a crazy context-dependent string/bytes type. Cool. In reality this would have caused endless confusion, especially with beginners and the scientific community, and likely killed the language or at the very least made the language a shadow of what it is now. They made the right choice given the outcome. Anything else is armchair postulation that was discussed previously and outright rejected for obvious reasons.
    - dataflow 6y ago
      > It worked. The ecosystem is on py3. Because they're doing everything they can to force py2 to go away. It's not it's dying a natural death out of disuse. Exhibit A is everyone else in this post still wanting to use it. If you think strings "work" under py3, my guess is you've never had to deal with all the edge cases, especially across all the 3 major desktop platforms. Possibly because your applications are limited in scope. (You're definitely not writing general-purpose libraries that guarantee correctness for a wide variety of usage.) Most things Python treats as Unicode text by default (file contents, file paths, command-line arguments, stdio streams, etc.) are not guaranteed to be contain only Unicode. They can have invalid Unicode mixed into them, either accidentally or intentionally, breaking programs needlessly. As a small example, try these and compare: python2 -c "import sys; print('Your input was:'); print(sys.argv[1])" $'\x80' | xxd python3 -c "import sys; print('Your input was:'); print(sys.argv[1])" $'\x80' | xxd This program is content-agnostic (like `cat`, `printf`, etc.), and hence, with a decent standard library implementation, you would expect it to be able to pass arbitrary data through just fine. But it doesn't, because Python insists on treating arguments as Unicode strings rather than as raw data, and it behaves worse on Python 3 than Python 2. You really have to go out of your way to make it work correctly—and the solution is often pretty much to just ditch strings in many places and deal with bytes as much as possible... i.e., you realize Unicode strings were the wrong data type. But since you're still forced to deal with them in some ways, you get the worst of both worlds and that increases the complexity dramatically and it become increasingly painful to ensure your program still works correctly as it evolves. I say all these because I've run into these and dealt with them, and it's become clear to me that others who love Unicode strings just haven't gone very far in trying to use them. Often this seems to be because they (a) are writing limited-scope programs rather than libraries, (b) confine themselves to nice, sanitized systems & inputs, and/or (c) take an "out-of-sight -> out-of-mind" attitude towards issues that don't immediately crop up on their systems & inputs.
IshKebab 6y ago
Yeah even CMake got the bit right. CMake!!

orf 6y ago

> All functions that return `bytes` continue to do so unless specifically opted in on a per file basis, then they return `unicode`. Nothing in py2 returns bytes. They all return strings. That is the issue. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > They would obviously not be removed and still be available but depræcated. Having two almost separate object models in the same language is rather silly.

Blikkentrekker 6y ago
> Nothing in py2 returns bytes. They all return strings. That is the issue. No, that is not an issue, that is semantics. What one calls it does not change the behavior. And aside that the system could perfectly well be designed that this pragma changes that `str` is synonymous with either `bytes` or `unicode` depending on it's state. What about subclasses or type wrappers? What about functions that return bytes or utf8 strings? How would you handle code that then calls “.startswith()” on a returned string/bytes value? You would now which is which by using the pragma or not. Not using the pragma defaults to the old behavior, as said, one only receives the new, breaking behavior, when one opts in. Python could even support always opting in by a configuration file option for those that really want it and don't want to add the pragma at the top of every file. > A language pragma that fundamentally alters a built in type across all the code you have in a program is never going to work and pushes the burden onto library authors to support a large matrix of different behaviours and types. Opposed to the burden they already had of maintaining a 2 and 3 version? Any new code can of course always return `unicode` rather than `str` which in this scheme is normally `bytes` but becomes `unicode` with the pragma. It would make the already ridiculous py2 str/bytes situation even more ridiculous. > Having two almost separate object models in the same language is rather silly. Yes, it is, and you will find that most languages are full of such legacy things that no new code uses but are simply for legacy purposes. “It is silly.” turns out to be a rather small price to pay to achieve. “We have not broken backwards compatibility.”.
- orf 6y ago
  I don’t really have the time of inclination to continue arguing, but I will point out that you say all this as though the approach the team took failed. It worked. The ecosystem is on py3. You can imagine some world with a crazy context-dependent string/bytes type. Cool. In reality this would have caused endless confusion, especially with beginners and the scientific community, and likely killed the language or at the very least made the language a shadow of what it is now. They made the right choice given the outcome. Anything else is armchair postulation that was discussed previously and outright rejected for obvious reasons.
  - dataflow 6y ago
    > It worked. The ecosystem is on py3. Because they're doing everything they can to force py2 to go away. It's not it's dying a natural death out of disuse. Exhibit A is everyone else in this post still wanting to use it. If you think strings "work" under py3, my guess is you've never had to deal with all the edge cases, especially across all the 3 major desktop platforms. Possibly because your applications are limited in scope. (You're definitely not writing general-purpose libraries that guarantee correctness for a wide variety of usage.) Most things Python treats as Unicode text by default (file contents, file paths, command-line arguments, stdio streams, etc.) are not guaranteed to be contain only Unicode. They can have invalid Unicode mixed into them, either accidentally or intentionally, breaking programs needlessly. As a small example, try these and compare: python2 -c "import sys; print('Your input was:'); print(sys.argv[1])" $'\x80' | xxd python3 -c "import sys; print('Your input was:'); print(sys.argv[1])" $'\x80' | xxd This program is content-agnostic (like `cat`, `printf`, etc.), and hence, with a decent standard library implementation, you would expect it to be able to pass arbitrary data through just fine. But it doesn't, because Python insists on treating arguments as Unicode strings rather than as raw data, and it behaves worse on Python 3 than Python 2. You really have to go out of your way to make it work correctly—and the solution is often pretty much to just ditch strings in many places and deal with bytes as much as possible... i.e., you realize Unicode strings were the wrong data type. But since you're still forced to deal with them in some ways, you get the worst of both worlds and that increases the complexity dramatically and it become increasingly painful to ensure your program still works correctly as it evolves. I say all these because I've run into these and dealt with them, and it's become clear to me that others who love Unicode strings just haven't gone very far in trying to use them. Often this seems to be because they (a) are writing limited-scope programs rather than libraries, (b) confine themselves to nice, sanitized systems & inputs, and/or (c) take an "out-of-sight -> out-of-mind" attitude towards issues that don't immediately crop up on their systems & inputs.

IshKebab 6y ago

Yeah even CMake got the bit right. CMake!!