4 ms·
CPU overhead is nowadays much less important than cache misses. UTF-8 has the least cache misses, Linux/BSD UTF-32 widechars the most, and Windows UCS-2 in betw
by rurban 7y ago
CPU overhead is nowadays much less important than cache misses. UTF-8 has the least cache misses, Linux/BSD UTF-32 widechars the most, and Windows UCS-2 in between. UTF-32 wchar_t was a gigantic design failure and should have been deprecated a long time ago. There's still no POSIX u8 api in sight.
The branch misses are not that high or costly, as most characters don't need the subsequent branches for 2-4 bytes, but cache misses cost ~50x more than a branch miss. And there are enough parallel pipelines now to avoid most utf-8 branch misses.
The new massive parallel SIMD ways to process UTF-8 are so fast because the width is so high, 256 bytes per loop, which is enough for most strings. But avoiding branching has some constant overhead to prepare the loops, for small strings it rarely pays off.