3 ms·
> Encoding system is beyond broken I disagree. Due to some political issues[1], in Ruby's country of origin, Unicode is still not as widely used as it could be
by pilif 14y ago
> Encoding system is beyond broken
I disagree. Due to some political issues[1], in Ruby's country of origin, Unicode is still not as widely used as it could be. Instead, local multibyte character sets are the norm.
Normalizing to Unicode internally (like what Python 3 does) is impractical or not feasible as turning that Unicode data back into the original string is not reliably possible but might be required.
Ignoring encodings all together and treating everything as byte strings (like 1.8.7 did) might be ok if you live in an english speaking country and you're not dealing with EBDIC. In all other cases, this means that you can't safely use any of the string manipulation built into the language.
As such the second best way to deal with encodings is to have a distinct type for every possible encoding you support and have them incompatible with each other.
Then the applications can decide what they want to do: Not deal with encodings at all (if you live in pure-ASCII land), unify everything to UTF-8 (or 16), or use whatever local character set the input data is in.
The only thing I would be willing to argue might have been a wrong decision is adding the force_encoding method as more often than not calling that is not what you want, even though it might look as if it was. It has a huge shooting-your-own-foot potential.
OTOH, sometimes libraries have mistakes or lie willingly at which point it might be a handy, albeit really dangerous, tool to have. Just like the -f flag for rm. Use it responsibly.
1) http://en.wikipedia.org/wiki/Han_unification http://en.wikipedia.org/wiki/Han_unification
- Xylakant 14y agocall it "cultural" instead of "political" and you're on the mark. force_encoding! is sometimes needed if you get data from inputs that pretend to be a different encoding than they actually are. It does offer a lot of rope to hang yourself with though.
- halostatue 14y agoIt's a bit of both. It's cultural insensitivity at the time it was done that resulted in the better part of a decade where Unicode was stalled in the East.