4 ms·

On a similar note, both Java and C# made an early and mostly incorrect decision to support utf-16 in the native string and char types. This choice also made som

On a similar note, both Java and C# made an early and mostly incorrect decision to support utf-16 in the native string and char types. This choice also made some programmers to believe that every unicode character fits in one char, causing bugs, etc. Yet, I have not seen a rush to "fix" any of them the Python way.

takeda 6y ago
You are mixing the internal representation of text with the change to make distinction between text and bytes. The python incompatibility wasn't really the Unicode it was that older versions of python mixed text with bytes, and because of that a lot of python 2 code is actually broken. It might run fine, but as soon as input are Unicode characters then many python 2 applications crash. Python 2 tried to implement Unicode and implemented an `unicode` type. But it actually made things worse due to them adding implicit casting. The proper way to fix the issue was actually clearly making the distinction what is text and what are bytes, something that was in both Java, C# (both of them are younger than Python BTW, the Unicode was in works when Python was released). he UTF-16 used by Java and C# as I said is just an internal representation of the text, and that's actually trivially changed since normally those things are not accessible for developers. The only way where it could somewhat leak is default encoding, and maybe names of the methods from stdlib. But those things are really easy to change without much breakage. Side note, Python internally stores text in many different ways if the text is only out bytes it uses traditional C-string (for performance). Otherwise it also stores as UTF-8, UTF-16, UTF-32 based on characters and requested conversion. So those conversion you said Java or C# did Python 3 does all the time (based on need) without breaking running code.
- veselin 6y ago
  I don't think I am mixing it up. The char type is not a unicode character just like the byte is not. Yet, these are the characters in a string type. Python went to fix this, jvm and clr didn't.
  - takeda 6y ago
    I'm not that familiar with C#[1], but in Java char is absolutely an unicode character. That's why you have char and byte types. Java clearly separates them, and so is python since version 3. [1] although from quick googling it looks like C# handles things the same way as Java.
    - veselin 6y ago
      This used to be the case when every unicode character could fit in 16 bits. Now, most methods that take char may fail: https://docs.oracle.com/javase/7/docs/api/java/lang/Character.html https://docs.oracle.com/javase/7/docs/api/java/lang/Characte...

takeda 6y ago

You are mixing the internal representation of text with the change to make distinction between text and bytes. The python incompatibility wasn't really the Unicode it was that older versions of python mixed text with bytes, and because of that a lot of python 2 code is actually broken. It might run fine, but as soon as input are Unicode characters then many python 2 applications crash. Python 2 tried to implement Unicode and implemented an `unicode` type. But it actually made things worse due to them adding implicit casting. The proper way to fix the issue was actually clearly making the distinction what is text and what are bytes, something that was in both Java, C# (both of them are younger than Python BTW, the Unicode was in works when Python was released). he UTF-16 used by Java and C# as I said is just an internal representation of the text, and that's actually trivially changed since normally those things are not accessible for developers. The only way where it could somewhat leak is default encoding, and maybe names of the methods from stdlib. But those things are really easy to change without much breakage. Side note, Python internally stores text in many different ways if the text is only out bytes it uses traditional C-string (for performance). Otherwise it also stores as UTF-8, UTF-16, UTF-32 based on characters and requested conversion. So those conversion you said Java or C# did Python 3 does all the time (based on need) without breaking running code.

veselin 6y ago
I don't think I am mixing it up. The char type is not a unicode character just like the byte is not. Yet, these are the characters in a string type. Python went to fix this, jvm and clr didn't.
- takeda 6y ago
  I'm not that familiar with C#[1], but in Java char is absolutely an unicode character. That's why you have char and byte types. Java clearly separates them, and so is python since version 3. [1] although from quick googling it looks like C# handles things the same way as Java.
  - veselin 6y ago
    This used to be the case when every unicode character could fit in 16 bits. Now, most methods that take char may fail: https://docs.oracle.com/javase/7/docs/api/java/lang/Character.html https://docs.oracle.com/javase/7/docs/api/java/lang/Characte...