3 ms·
Since values 0-127 are used far more frequently than 128-255 in latin-1, it might make more sense to simply have a fast path which simply loads 512 bits at a ti
by londons_explore 3y ago
Since values 0-127 are used far more frequently than 128-255 in latin-1, it might make more sense to simply have a fast path which simply loads 512 bits at a time (ie. 64 bytes), detects if any are 0x80 or above, and if not just outputs them verbatim.
- jojobas 3y agoEither way throughput will depend on the fraction of >192 characters, what input data gave 12GB/s seems to be a mystery.
- reaperhulk 3y agoThe article states it's the French version of the Mars wikipedia entry and the repository has a link to the file he used in the readme: https://raw.githubusercontent.com/lemire/unicode_lipsum/main/wikipedia_mars/french.latin1.txt https://raw.githubusercontent.com/lemire/unicode_lipsum/main...
- deleted 3y ago[deleted]
- twoodfin 3y agoI don’t know if the article has been updated since your comment, but this approach is discussed & benchmarked. For the benchmarked data set it’s a winner.
- wffurr 3y agoThe article was indeed updated since I read it and the parent comment this morning.
- NelsonMinar 3y agoThe article has a whole section about that, you might enjoy reading about it. He reports a ~20% speedup on his test data.