3 ms·
I don't usually want to remove all whitespace from text, but I do often want to remove weird Unicode whitespace characters (there are dozens of them), control c
by kijin 8y ago
I don't usually want to remove all whitespace from text, but I do often want to remove weird Unicode whitespace characters (there are dozens of them), control characters, excessively repeated diacritics, and unbalanced bidirectional markers from user-submitted content.
The code will have to be a lot more complicated in order to achieve this, and I'm not even sure I could use AVX512 to replace the ugly regex I'm currently relying on. But the idea is close enough, so maybe whitespace removal is meant to serve as a simple proof of concept.
- vardump 8y agoWhen your starting point is a regexp, I think you have a few layers of lesser abstraction to explore for optimization opportunities before trying to implement something in AVX512...