4 ms·
That's really cool! Character encoding issues is something we wrestle with all the time, and it is surprisingly hard to reason about all the ways supposedly "st
by drunkpotato 13y ago
That's really cool! Character encoding issues is something we wrestle with all the time, and it is surprisingly hard to reason about all the ways supposedly "string" data are handled in the course of a typical workflow. I cringe; I hadn't even considered bugs in the encoding and decoding process itself.
- hannibal5 13y agoHow many open source libraries and programs there exist that can actually work correctly with full Unicode with all kinks involved? I know none. Emacs seems to do best job, but I have not investigated it much. I know one proprietary library for Unicode that can be used for search (from multiple different sources of UTF strings), indexing etc. and claims to support full Unicode that deals with all things involved, including directionality, surrogates, control chars etc. Their internal representation of stings for string processing is vector of displayed characters objects (not code points). Using UTF-* encoding as internal representation works only for simple string processing for subset of Unicode.