7 ms·
It is worth noting that the original reason for Ruby is that 20 years ago there was no scripting language which could handle Japanese text. Hence a Lisp hacker
by btilly 9y ago
It is worth noting that the original reason for Ruby is that 20 years ago there was no scripting language which could handle Japanese text.
Hence a Lisp hacker decided to write a scripting language which could do so using the Smalltalk object model with a library carefully designed to would work a lot like Perl. The result was Ruby.
- euske 9y agoBlatantly false. Perl4 could handle Japanese text perfectly okay in early 90s. Neither Perl or Ruby supported character encoding initially and you had to rely on NKF/KConv etc.
- wodenokoto 9y agoBeing able to handle Japanese and being able to handle character encodings are two very different things. Python 2.7 (byte) strings handle Japanese characters just fine for a lot of cases. I don't think you nor parent have made a clear point for either case
- chongli 9y agoPython 2.7 (byte) strings handle Japanese characters just fine for a lot of cases. Python 2.7 was released in 2010, 15 years after the release of Ruby. At the time of Ruby's release, you were looking at Python 1.0.
- masklinn 9y agoAnd Python 1.0 handled text (in general) in the exact same way Ruby did: by ignoring it altogether and just manipulating bags of bytes.
- fenomas 9y ago> no scripting language which could handle Japanese text. Matz doesn't say this anywhere I can find. He usually says he wanted a more object-oriented scripting language and wasn't satisfied with Perl or Python.
- paulddraper 9y agoIf that's true, it's surprising then that Ruby handles encodings so badly. String encoding cause some pretty obtuse errors in non-obvious spots. irb(main):001:0> str = "\xAA" => "\xAA" irb(main):002:0> str.split('') ArgumentError: invalid byte sequence in UTF-8 from (irb):2:in `split' from (irb):2 from /usr/bin/irb:12:in `<main>' WTF?
- bhaak 9y agoFirst, you're looking at a feature of modern Ruby that got added in 1.9. Before that, Ruby only supported a single encoding. If you're interesting in the details, look up how the $KCODE global variable used to affect in Ruby <= 1.8. Second, what do you expect if you have an invalid byte sequence in your UTF-8 string? str.valid_encoding? would have told you that the string contains invalid characters. String#encode gives you fine grained control over how to handle invalid characters in your strings. Ruby lets you set the encoding by string. > str = "\xa4" > puts str � > puts str.force_encoding("iso-8859-1").encode("utf-8") ¤ > puts str.force_encoding("iso-8859-16").encode("utf-8") €
- lloeki 9y agoYou just made the point of the GP, Ruby does handle encoding, but it does so badly: > str = "\xa4" "" creates a UTF8 string, that one happens to contain an invalid char sequence right off the bat > str.split('') ArgumentError: invalid byte sequence in UTF-8 ... which can only be detected at the worst possible time: when processed at runtime > puts str � ... and not even always, as sometimes ruby decides to handle strings as just bytes > str.force_encoding("iso-8859-1").encode("utf-8") ... and leaves the responsibility of whether they're bytes or strings, (and which encoding of) solely on the programmer. "force_encoding" is a thing that just should not exist on a string. Python 3 and Go do it the correct way: array of bytes are array of bytes and strings are {array of bytes + encoding}. Ruby could have done it too by introducing a new type or something but chose to do otherwise in the name of a theoretical backwards compatibility that doesn't exist in practice since every singe release of Ruby introduced backwards-incompatible changes anyway.