5 ms·
It's important to distinguish between sorting in code point order and sorting according what a user would expect for their language. However, sorting in code p
by rectang 6y ago
It's important to distinguish between sorting in code point order and sorting according what a user would expect for their language. However, sorting in code point order (which is actually equivalent to sorting by memory comparison for UTF-8) is enough to build an inverted index data structure, commonly used for fulltext search. And just like FridgeSeal asserted, the memory footprint of the text representation has performance implications for such an application.
(Source: I wrote a search engine library.)
- tsimionescu 6y agoWill that handle things like matching accented and unaccented characters? If I search for 'stefan' in a text that makes frequent references to 'Ștefan', will it correctly find those matches?