3 ms·
This page has the best implementation I've seen: http://www.daemonology.net/blog/2008-06-05-faster-utf8-strlen.html http://www.daemonology.net/blog/2008-06-05-f
by elFarto 11y ago
This page has the best implementation I've seen: http://www.daemonology.net/blog/2008-06-05-faster-utf8-strlen.html http://www.daemonology.net/blog/2008-06-05-faster-utf8-strle...
It basically counts continuation bytes (which all start 10xxxxxx) and substracts, rather than trying to count characters.
Additionally, if you know how many bytes are in the string, you can remove the check for the null terminator.
- vardump 11y agoTested my idea quickly. Initial messy (but correct) version finishes in 31% of time that version you linked [1] (cp_strlen_utf8) takes to run. I think I can still improve it quite a bit. Only tested with long 30 MB strings. Edit: Now at 26%. But it can still be improved more. Both benchmarks are with hot cache. new_strlen_utf8 12352856 clock cycles cp_strlen_utf8 47544818 clock cycles Edit 2: Well, 4x performance 32 bit, but compiling it 64-bit in VS2015RC manages to optimize cp_strlen_utf8 more, almost doubling performance. 45% then. Will try gcc 5, clang, etc. later. And it can still be optimized further. Edit 3: Ended up at 35% (2.9x) execution time for 64-bit and 18% (5.5x) for 32-bit. My version is as fast in 32 and 64-bit, but cp_strlen_utf8 benefits quite a bit from 64-bit mode. Probably memory bandwidth limited at this point, but I didn't profile yet. In any case, it does utf-8 code point strlen at 16 GB/s at this point. CPU is i5-4430 CPU @ 3.00GHz, two memory channels @1600 MHz. [1]: http://www.daemonology.net/blog/2008-06-05-faster-utf8-strlen.html http://www.daemonology.net/blog/2008-06-05-faster-utf8-strle...