4 ms·
As this rebuttal points out, the original article basically finds two flaws in Python 3 and then tries to puff them up as much as possible. These two complaints
by jcranmer 10y ago
As this rebuttal points out, the original article basically finds two flaws in Python 3 and then tries to puff them up as much as possible. These two complaints are:
* Python 3 is not backwards-compatible with Python 2.
* Byte strings and character strings are different things so now I have to care about stuff.
Although I say that it's two things, it's really one thing since the main element of incompatibility is the byte string/character string issue, since it's that incompatibility that makes it hard to know what the right thing to do is.
Having had to deal with internationalization before, I can honestly say that there's only one way to handle it: convert all string data to your internal representation (be it UTF-8 (Rust), UTF-16 (Java), or UTF-32 (Python)) at the margin where you read/write it, and keep binary strictly separate from textual in terms of representation. This is what Python 3 does and what Python 2 doesn't do. I have seen some people argue that Python 2's model is better because it means that writing something like cat is harder, but that's optimizing for the wrong thing. I've seen a build system written in Python 2 break for the sole reason that I thought to write 一 in the commit message.
As for the complaint that it's taking 9 years for Python3 to slowly become the default instead of Python2, keep in mind that 15 years after the release of C99, gcc was still using C89 by default (gcc 5 skipped C99 and went straight to C11).
- the_mitsuhiko 10y ago> Having had to deal with internationalization before, I can honestly say that there's only one way to handle it: convert all string data to your internal representation (be it UTF-8 (Rust), UTF-16 (Java), or UTF-32 (Python)) at the margin where you read/write it I'm not going to go into the actual subtleties here but there is a lot more to it than what you suggest here. However the biggest mistake you made here is to equate that any encoding is equivalent here. There is a tremendous difference between using utf-8 as internal encoding or a variable encoding like in Python 3. Rust and Go have free bytes/string conversion wheres Python 3 has none of that.
- jcranmer 10y agoPython 3's string types are internally ASCII (or Latin-1, I'm not sure), UTF-16, or UTF-32 depending on the contents of the string, but the API effectively assumes that the encoding is UTF-32. The main point is that you don't want to be passing around binary data internally and expecting people to decode charsets as needed--the knowledge of charsets will usually be dropped.
- the_mitsuhiko 10y ago> Python 3's string types are internally ASCII (or Latin-1, I'm not sure), UTF-16, or UTF-32 depending on the contents of the string, but the API effectively assumes that the encoding is UTF-32. That is incorrect. The API assumes the string is O(1) indexable. It makes no assumptions or guarantees about the internal encoding. > The main point is that you don't want to be passing around binary data internally and expecting people to decode charsets as needed--the knowledge of charsets will usually be dropped. This is an incorrect blanket statement and I can give you plenty of examples where this is exactly what you will have to do because the knowledge of the encoding is not available until later. This is in fact a common problem that Python 3 has because it decodes to unicode so early that it had to come up with surrogate-escapes to roundtrip bytes in case the encoding it guessed/inferred was wrong.
- mattashii 10y agoHe's talking about CPython, the default interpreter/compiler for Python in many systems. CPython internally uses 1-byte representation if the maximum Unicode code point of its chars is 127 or less (therefore ASCII), 2-byte representation for values less than 32767 (thus utf-16, but only the first half-page), and for the rest it uses UTF-32, which means all lookups can be done in O(1). Assuming UTF-8 encoding for source files is not that strange, as it is getting more and more the default or preferred encoding for string-based I/O.
- the_mitsuhiko 10y agoI understand how CPython works however the external API makes no guarantees about any internal encoding. The implementation of the storage also changed many times.
- echlebek 10y agoDo you have a source for Go having free byte/string conversion? To my knowledge it is a copy. Or maybe I don't get your meaning? I really like the way Go's strings work. I wish Python's strings worked like this too. https://play.golang.org/p/a70FEDOxBz https://play.golang.org/p/a70FEDOxBz
- the_mitsuhiko 10y ago> Do you have a source for Go having free byte/string conversion? To my knowledge it is a copy. Or maybe I don't get your meaning? You can access bytes in go out of the string without even having to perform a conversion. For bytes to string you can use the unsafe and reflect package to create strings from bytes without a copy. (You will need to verify utf-8 integrity of course)
- gengkev 10y agoIn contrast to what Zed suggests, I would argue that the text/binary dichotomy that you describe is actually very important for beginners to learn. Until I learned Python 3, I never really understood how text encodings worked, which confused me whenever I had to deal with Unicode (I'm looking at you, UTF-16).
- kosma 10y ago> * Byte strings and character strings are different things so now I have to care about stuff. The vast majority of Python programmers who don't live in a mostly-ASCII world just breathed a sigh of relief. Python 3 makes Unicode awareness mandatory instead of optional; that's the right move for a web-facing, human-facing language.