5 ms·

I can tell that proper Unicode support is a reason to discounting a language/library if you're not a native English speaker and/or writing programs that have to

by gtufano 12y ago

I can tell that proper Unicode support is a reason to discounting a language/library if you're not a native English speaker and/or writing programs that have to be localized.

icebraining 12y ago
Python was created by a non-native English speaker.
- eloisant 12y ago
  Well, he's Dutch, his language uses an alphabet meaning that even with all the accents it's still a one byte language. Having to work with 2 bytes languages (CJK typically) without proper Unicode support is much more of a hassle.
  - gldsmth 12y ago
    > Having to work with 2 bytes languages (CJK typically) without proper Unicode support is much more of a hassle. Proper unicode support is not even close to being a priority when it comes to dealing with Japanese text. No one would store important Japanese text in unicode. http://en.wikipedia.org/wiki/Han_unification#Unihan_.22abstract_characters.22 http://en.wikipedia.org/wiki/Han_unification#Unihan_.22abstr... One size doesn't fit all.
  - thinkpad20 12y ago
    This makes me wonder about Ruby, whose creator is Japanese and Japanese is certainly not a language whose characters can be expressed in a single byte. Is Ruby better than Python for complex encodings? If so has it always been this way, or did Matz just deal with it in the early days?
    - lmm 12y ago
      Yes, Ruby has long had decent (i.e. comparable to Python 3) encoding support. But in the early days not much was made of this, perhaps because Rails was the driving force behind Ruby adoption in the west and Rails' encoding support was much worse than what Ruby could do.
- gtufano 12y ago
  I know that. I'm Italian, and Western European Languages can be expressed on a 1-byte character type. I also understand that the performance gain of 1 byte char was significant. But now is 2014 and not to have to wrestle with encodings definitely worth the performance and (somewhat) complexity hit. As a unrelated (to Python) example of the encoding nightmare: have you ever tried to import text from CSV files generated on different OS/locales in Excel that don't support Unicode-encoded CSV files? It can be made, sure... but, man, if it can be avoided... ;)
  - aidos 12y ago
    I did this recently. I can't remember the details but it was absolutely insane.
- rcpt 12y ago
  Before Unicode existed
- pekk 12y ago
  And Python3 was also endorsed by the same person, so?

by gtufano 12y ago

I can tell that proper Unicode support is a reason to discounting a language/library if you're not a native English speaker and/or writing programs that have to be localized.

icebraining 12y ago

Python was created by a non-native English speaker.

eloisant 12y ago
Well, he's Dutch, his language uses an alphabet meaning that even with all the accents it's still a one byte language. Having to work with 2 bytes languages (CJK typically) without proper Unicode support is much more of a hassle.
- gldsmth 12y ago
  > Having to work with 2 bytes languages (CJK typically) without proper Unicode support is much more of a hassle. Proper unicode support is not even close to being a priority when it comes to dealing with Japanese text. No one would store important Japanese text in unicode. http://en.wikipedia.org/wiki/Han_unification#Unihan_.22abstract_characters.22 http://en.wikipedia.org/wiki/Han_unification#Unihan_.22abstr... One size doesn't fit all.
- thinkpad20 12y ago
  This makes me wonder about Ruby, whose creator is Japanese and Japanese is certainly not a language whose characters can be expressed in a single byte. Is Ruby better than Python for complex encodings? If so has it always been this way, or did Matz just deal with it in the early days?
  - lmm 12y ago
    Yes, Ruby has long had decent (i.e. comparable to Python 3) encoding support. But in the early days not much was made of this, perhaps because Rails was the driving force behind Ruby adoption in the west and Rails' encoding support was much worse than what Ruby could do.
gtufano 12y ago
I know that. I'm Italian, and Western European Languages can be expressed on a 1-byte character type. I also understand that the performance gain of 1 byte char was significant. But now is 2014 and not to have to wrestle with encodings definitely worth the performance and (somewhat) complexity hit. As a unrelated (to Python) example of the encoding nightmare: have you ever tried to import text from CSV files generated on different OS/locales in Excel that don't support Unicode-encoded CSV files? It can be made, sure... but, man, if it can be avoided... ;)
- aidos 12y ago
  I did this recently. I can't remember the details but it was absolutely insane.
rcpt 12y ago
Before Unicode existed
pekk 12y ago
And Python3 was also endorsed by the same person, so?