5 ms·
You also get a lower clock rate from running scalar code on many cores at once. Use of the 512-bit unit only makes the coefficient different. That said, the b
by shereadsthenews 8y ago
You also get a lower clock rate from running scalar code on many cores at once. Use of the 512-bit unit only makes the coefficient different.
That said, the biggest mystery in this article is why you'd ever want to remove all whitespace from text. Why is that useful?
- taspeotis 8y agoI dunno, spam filtering? sig nup now for che ap viag ra And then search for key words like “viagra.”
- jasonzemos 8y agoIf one could remove spaces from JSON to create a so-called "Canonical JSON" one could obtain the same digest hash from the same data (even in combination with the hardware accelerated hashing offered by AVX!). Admittedly this is a strange case but I've run into it.
- jandrese 8y agoIsn't this a problem if you're fed these two inputs: {mystring:"Hello World!"} {mystring:"Hello World !"} They aren't the same JSON.
- marvy 8y agoalso: {mystring:"HelloWorld!"}
- amenghra 8y agoTypically, when you want a canonical JSON you don't want to touch strings and you want to sort maps.
- kijin 8y agoI don't usually want to remove all whitespace from text, but I do often want to remove weird Unicode whitespace characters (there are dozens of them), control characters, excessively repeated diacritics, and unbalanced bidirectional markers from user-submitted content. The code will have to be a lot more complicated in order to achieve this, and I'm not even sure I could use AVX512 to replace the ugly regex I'm currently relying on. But the idea is close enough, so maybe whitespace removal is meant to serve as a simple proof of concept.
- vardump 8y agoWhen your starting point is a regexp, I think you have a few layers of lesser abstraction to explore for optimization opportunities before trying to implement something in AVX512...
- pkaye 8y agoMaybe for the International Obfuscated C Code Contest.
- tgtweak 8y agoCompression routines regularly track then remove whitespace. Given the commonplace occurrence of both compression serverside and decompression clientside on the web, this improvement could be notable in every day computing.
- wmu 8y agoThis is a problem with base64 decoding. While there are vectorized algorithms for decoding and encoding, they do expect a text without spaces or newlines. However, by design, in emails the base64-encoded data must be split to lines not exceeding some length.