3 ms·
I suspect primarily substring, where if you index by bytes you'll mangle the string, but if you index by codepoints everything works out.
by insanitybit 3y ago
I suspect primarily substring, where if you index by bytes you'll mangle the string, but if you index by codepoints everything works out.
- deleted 3y ago[deleted]
- jlokier 3y agoYou'll mangle the string if you index and search by code points too, when the string contains the emoji this article is about, or for that matter an "e" following by combining acute accent. The string will be fine if you only move to, split and concatenate at indices outside those grapheme clusters. But that is also true when indexing by bytes or UTF-16 code units. So in some senses, indexing by bytes is just as good as indexing by code points, but faster. Either way to avoid mangling strings you need to restrict the indices of whatever type to meaningful character boundaries. If you have decided to avoid string indices inside grapheme clusters, there comes the awkard question of what should you do when editing text in an environment rendered with font ligatures like "->" rendered as → (rightward arrow). From one perspective, that's just a font. From another, the user sees a single character yet there are valid positions (such as from cursor movement and character search) that land mid-way through the character, and editing at those positions changes the character. Neither is clearly best for all situations.