2 ms·
There's no fixed-width/non-allocating way to return a grapheme (unless one can return a slice into the original data, which cannot happen here), because they ma
by dbaupp 8y ago
There's no fixed-width/non-allocating way to return a grapheme (unless one can return a slice into the original data, which cannot happen here), because they may consist of arbitrarily many codepoints (and code units) both in theory and in practice. For instance, many non-Latin scripts compose single graphemes out of multiple code points and many emoji are multiple codepoints (e.g. the families[0], and the skin tone variants).
https://manishearth.github.io/blog/2017/01/15/breaking-our-latin-1-assumptions/ https://manishearth.github.io/blog/2017/01/15/breaking-our-l...
One option would to instead return (24-bit) code points, which would amortize comparison and iteration costs slightly, but may accidentally encourage people to forget about multiple-codepoint characters.
[0]: https://r12a.github.io/uniview/?charlist=%F0%9F%91%A9%E2%80%8D%F0%9F%91%A9%E2%80%8D%F0%9F%91%A7%E2%80%8D%F0%9F%91%A6 https://r12a.github.io/uniview/?charlist=%F0%9F%91%A9%E2%80%...
- saagarjha 8y ago> There's no fixed-width/non-allocating way to return a grapheme (unless one can return a slice into the original data, which cannot happen here) I take it's not possible to return a pointer into the data because normalization is being performed?
- dbaupp 8y agoYeah, normalising and case folding will both potentially result in a new sequence of graphemes (and code points and code units), so slicing can't work.