4 ms·

Computing the UTF-8 size of a Latin 1 string quickly (ARM NEON edition)

• Someone 3y ago
  …if your strings - are long (code uses 16,000 bytes) - have a length that’s divisible by 64 (same reason). I haven’t checked it, but code coverage should show that the benchmark never runs the ”scalar tail” part that all versions share. - have contents with an uniform distribution of byte values (close to half of them are in the [128, 255] range. I suspect that doesn’t matter for these algorithms, but don’t know enough of the various micro-architectures to be sure about that. Also, the code weirdly reports speed in terms not of input, but of output size. Since it initializes the data with calls to rand, that means the reported speed may get twice as high if you’re extremely^extremely lucky. Finally, going deep into nitpick territory, would a hypothetical super-optimizing compiler could exploit known weaknesses in rand (typically a linear congruential generator (https://en.wikipedia.org/wiki/Linear_congruential_generator https://en.wikipedia.org/wiki/Linear_congruential_generator)), and almost completely remove the length computations? I think it is, by the standard, allowed to assume to know what exact generator that uses (C and C++ compilers are somewhat tied to their standard libraries)