10 ms·

No, because they did a bunch of things that make automated porting not work well and provides a ton of developer friction. I'm willing to live with Python's sp

No, because they did a bunch of things that make automated porting not work well and provides a ton of developer friction. I'm willing to live with Python's space related insanity. Mandating that indenting is meaningful. Mandating a best practice of using spaces instead of tabs for indent* (which makes tab width matter). Mandating that a tab equals 4 spaces instead of 8. (Honestly, I REALLY hate this, most of the Windows developers are happy with the space usage koolaid, let us have a default of tab equals 8 spaces of indent.) Those are ALL, REALLY ANNOYING, but lint tools can save me. What will bite me for as long as Python 3 exists, and maybe Python 4, is how they handle Unicode support and encoded outputs. Pretty much the ENTIRE rest of the computing world does one of two things. 1) Doesn't care what the encoding is, and flings around non-validated 8 bit sequences of data. 2) Does care, and defaults to assuming well formed UTF-8 data-streams for text data, BUT still doesn't do anything to enforce it UNLESS ASKED. Most importantly invalid data streams just continue flowing through the processing tools until they reach a point where a human has designated they're willing to care and /do/ something about it. If Python 4 wants to 'fix' Unicode support: * Use a binary sequence base storage class with metadata tags. > Length (in bytes, in 'display length', and maybe in 'codepoint length') > encoding* (including if normalized and how so) > If the encoding has been validated. * Support automatic coercion between standard encodings * Allow user defined helpers for custom conversions * Make all input/output default to UTF-8** but permit raw output of such types without validation or re-encoding, just like every other UNIX tool that knows what an encoding is.

singularity2001 9y ago
Yes, please make UTF-8 default! It's 2018 print(str(b'I DONT WANT TO add "UTF-8" everywhere!!','UTF-8'))
- ubernostrum 9y ago
  As a serialization and transmission format, UTF-8 is fine. As the basis of a high-level language's string type, UTF-8 is objectively incorrect. Strings in a high-level language should be as clean an abstraction of Unicode as possible, and leaking implementation details of the particular byte-encoding scheme up to the programmer is not acceptable.
  - singularity2001 9y ago
    Ok, is it reasonable to ask for all string getter/setter file read/write ops to default to UTF-8? also having different string methods .byte_length .char_length(?) .codepoint_length seems a good idea. or string.length(aspect=bytes) but what should be the default aspect?
    - mjevans 9y ago
      For compatibility reasons, len(unicode type) should probably return the size if accessed as a raw 8 bit sequence, so an alias to .length_bytes. I also think the other names should similarly start with .length_ .
      - mark-r 9y ago
        len() should return the length in individual addressable units, so that obj[len(obj)-1] always returns the last item, whether that's a byte or a unicode character.
    - ubernostrum 9y ago
      I'm going to quote something I wrote a couple years ago[1]: Now, I should point out here that I’m not really knocking the people who were writing, say, command-line and file-handling utilities in Python. For years, Python sort of accepted the status quo of the Unix world, which was mostly to stick its fingers in its ears and shout LA LA LA I CAN’T HEAR YOU I’M JUST GOING TO SET LC_CTYPE TO C AGAIN AND GO BACK TO MY HAPPY PLACE. A bit later on it changed to “just use UTF-8 everywhere, UTF-8 is perfectly safe”, which really meant “just use UTF-8 everywhere because we can continue pretending it’s ASCII up until the time someone gives us a non-ASCII or multi-byte character, at which point do the fingers-in-ears-can’t-hear-you thing again”. So a lot of what you’ll see in terms of complaints about string handling are really complaints that Unix’s pretend-everything-is-ASCII-until-it-breaks approach was never very good to begin with and just gets worse with every passing year. I stand by this: we had a couple of decades of Python catering to this brokenness, and it made life miserable for everyone who didn't work in that particular domain. Python 3 changed that. Does it mean life got harder for some people? Yup. But life got a lot easier and more reliable for many more people, and it's a tradeoff I'm willing to accept. [1] https://www.b-list.org/weblog/2016/jun/10/python-3-again/ https://www.b-list.org/weblog/2016/jun/10/python-3-again/
- viraptor 9y ago
  Not sure what you mean. It's the default for a long time now. https://docs.python.org/3/library/stdtypes.html#bytes.decode https://docs.python.org/3/library/stdtypes.html#bytes.decode `open` also defaults to your system encoding, which is likely utf8 judging from the message.
  - singularity2001 9y ago
    print(str(b'I am a broken string')) print(str(b'I DONT WANT TO add "UTF-8" everywhere!!','UTF-8')) not quite default!
    - orf 9y ago
      Explicit is better than implicit. The first line isnt a string, it's a collection of bytes, guessing that it's utf8 is not the right thing to do. The encode (or is it decode? I always forget) defaults to utf8, which is what you should be using.
      - Siecje 9y ago
        It helps me to think encrypt for encode (convert to bytes) and decrypt for decode (convert to text).
    - viraptor 9y ago
      I'm not sure why this example is interesting. You can write: print('I DONT WANT TO add "UTF-8" everywhere!!') And get the same result. Or if you really want to start with bytes: print(b'some string'.decode())
polpo 9y ago
A literal tab character is equivalent to 8 spaces in Python source (or 1-7 spaces if the tab is preceded by a number of spaces that is not an even multiple of 8). This follows well-established Unix standards. So the following lines are at equivalent indentation: <SPC><SPC><SPC><SPC><SPC><SPC><SPC><SPC>foo <TAB>foo Or do you mean that when you press the tab key in your editor when editing Python, you don't like that it emits 4 spaces instead?
viraptor 9y ago
I think this world exists in Ruby and as far as I can tell it fails worse in comparison. You end up in situations where 2 layers of gems don't care what they're passing through and operating on, then you get the result and can't tell anything useful about the contents. Was the encoding preserved? Is it even encoded? People end up using hacks like https://github.com/whitequark/rack-utf8_sanitizer https://github.com/whitequark/rack-utf8_sanitizer which shouldn't be necessary in the first place, because you should know whether you're receiving raw bytes or text. But what you thought was text that's utf8-compatible actually includes characters from another encoding. `String.valid_encoding?` is probably the worst programming idea. If you have to ask that, it's too late. There should've been an exception raised when the input was initially processed, or the type should not pretend that it's a real text.

singularity2001 9y ago

Yes, please make UTF-8 default! It's 2018 print(str(b'I DONT WANT TO add "UTF-8" everywhere!!','UTF-8'))

ubernostrum 9y ago
As a serialization and transmission format, UTF-8 is fine. As the basis of a high-level language's string type, UTF-8 is objectively incorrect. Strings in a high-level language should be as clean an abstraction of Unicode as possible, and leaking implementation details of the particular byte-encoding scheme up to the programmer is not acceptable.
- singularity2001 9y ago
  Ok, is it reasonable to ask for all string getter/setter file read/write ops to default to UTF-8? also having different string methods .byte_length .char_length(?) .codepoint_length seems a good idea. or string.length(aspect=bytes) but what should be the default aspect?
  - mjevans 9y ago
    For compatibility reasons, len(unicode type) should probably return the size if accessed as a raw 8 bit sequence, so an alias to .length_bytes. I also think the other names should similarly start with .length_ .
    - mark-r 9y ago
      len() should return the length in individual addressable units, so that obj[len(obj)-1] always returns the last item, whether that's a byte or a unicode character.
  - ubernostrum 9y ago
    I'm going to quote something I wrote a couple years ago[1]: Now, I should point out here that I’m not really knocking the people who were writing, say, command-line and file-handling utilities in Python. For years, Python sort of accepted the status quo of the Unix world, which was mostly to stick its fingers in its ears and shout LA LA LA I CAN’T HEAR YOU I’M JUST GOING TO SET LC_CTYPE TO C AGAIN AND GO BACK TO MY HAPPY PLACE. A bit later on it changed to “just use UTF-8 everywhere, UTF-8 is perfectly safe”, which really meant “just use UTF-8 everywhere because we can continue pretending it’s ASCII up until the time someone gives us a non-ASCII or multi-byte character, at which point do the fingers-in-ears-can’t-hear-you thing again”. So a lot of what you’ll see in terms of complaints about string handling are really complaints that Unix’s pretend-everything-is-ASCII-until-it-breaks approach was never very good to begin with and just gets worse with every passing year. I stand by this: we had a couple of decades of Python catering to this brokenness, and it made life miserable for everyone who didn't work in that particular domain. Python 3 changed that. Does it mean life got harder for some people? Yup. But life got a lot easier and more reliable for many more people, and it's a tradeoff I'm willing to accept. [1] https://www.b-list.org/weblog/2016/jun/10/python-3-again/ https://www.b-list.org/weblog/2016/jun/10/python-3-again/
viraptor 9y ago
Not sure what you mean. It's the default for a long time now. https://docs.python.org/3/library/stdtypes.html#bytes.decode https://docs.python.org/3/library/stdtypes.html#bytes.decode `open` also defaults to your system encoding, which is likely utf8 judging from the message.
- singularity2001 9y ago
  print(str(b'I am a broken string')) print(str(b'I DONT WANT TO add "UTF-8" everywhere!!','UTF-8')) not quite default!
  - orf 9y ago
    Explicit is better than implicit. The first line isnt a string, it's a collection of bytes, guessing that it's utf8 is not the right thing to do. The encode (or is it decode? I always forget) defaults to utf8, which is what you should be using.
    - Siecje 9y ago
      It helps me to think encrypt for encode (convert to bytes) and decrypt for decode (convert to text).
  - viraptor 9y ago
    I'm not sure why this example is interesting. You can write: print('I DONT WANT TO add "UTF-8" everywhere!!') And get the same result. Or if you really want to start with bytes: print(b'some string'.decode())

polpo 9y ago

A literal tab character is equivalent to 8 spaces in Python source (or 1-7 spaces if the tab is preceded by a number of spaces that is not an even multiple of 8). This follows well-established Unix standards. So the following lines are at equivalent indentation: <SPC><SPC><SPC><SPC><SPC><SPC><SPC><SPC>foo <TAB>foo Or do you mean that when you press the tab key in your editor when editing Python, you don't like that it emits 4 spaces instead?

viraptor 9y ago

I think this world exists in Ruby and as far as I can tell it fails worse in comparison. You end up in situations where 2 layers of gems don't care what they're passing through and operating on, then you get the result and can't tell anything useful about the contents. Was the encoding preserved? Is it even encoded? People end up using hacks like https://github.com/whitequark/rack-utf8_sanitizer https://github.com/whitequark/rack-utf8_sanitizer which shouldn't be necessary in the first place, because you should know whether you're receiving raw bytes or text. But what you thought was text that's utf8-compatible actually includes characters from another encoding. `String.valid_encoding?` is probably the worst programming idea. If you have to ask that, it's too late. There should've been an exception raised when the input was initially processed, or the type should not pretend that it's a real text.