3 ms·
This should be in the standard library, but maybe I'm biased. In implementing ascii/utf8 plain text protocols like IMAP/SMTP just like the grep/ripgrep example
by epilys 4y ago
This should be in the standard library, but maybe I'm biased. In implementing ascii/utf8 plain text protocols like IMAP/SMTP just like the grep/ripgrep example in the article, I've had to limit myself to u8 slices just because the occasional byte might make a grapheme invalid.
- burntsushi 4y agoYeah people say this enough that I added a section about it to the blog: https://blog.burntsushi.net/bstr/#should-byte-strings-be-added-to-std https://blog.burntsushi.net/bstr/#should-byte-strings-be-add...
- epilys 4y agoThe needle/haystack thing is the crucial part as you say in the blog. Since grapheme, word separation and sentence segmentation rely on rules that are updated on each unicode version it indeed would be out of place in the standard library. Ideally I'd want them (libicu) on the system level, e.g. standardised in POSIX in libc so that language developers shouldn't need to worry about them.