3 ms·
> It's boundary-stuff, and none of your actual text-processing should depend on it. I strongly disagree. For high performance text search, it's critical that y
by burntsushi 9y ago
> It's boundary-stuff, and none of your actual text-processing should depend on it.
I strongly disagree. For high performance text search, it's critical that you deal with its in-memory representation explicitly. This violates your maxim that such things are only done at the boundaries.
For example, if you're implementing substring search, the techniques you use will heavily depend on how your string is represented in memory. Is it UTF-16? UTF-8? A sequence of codepoints? A sequence of grapheme clusters, where each cluster is a sequence of codepoints? Each of these choices will require different substring search strategies if you care about squeezing the most juice out of the underlying hardware.
- ademarre 9y agoSometimes I think we would be better off if our languages didn't have string as a data type at all. The text encodings themselves (e.g. UTF-8, UTF-32) ought to be proper data types. Strings are a leaky abstraction that cause otherwise competent programmers to have funny ideas about what text is and isn't, as this entire thread demonstrates.