4 ms·
I’ve been dealing with an example of just what you described: Unicode normalization and collation. There’s a good normalization library in Rust—in terms of the
by theobeers 4y ago
I’ve been dealing with an example of just what you described: Unicode normalization and collation. There’s a good normalization library in Rust—in terms of the quality of the code, standards conformance, how comfortable I am depending on it, etc.—but it’s a full order of magnitude slower than icu4c.[0] Which shouldn’t come as a surprise! Some of these things take decades.
[0]: https://github.com/unicode-org/icu4x/issues/93#issuecomment-631102136 https://github.com/unicode-org/icu4x/issues/93#issuecomment-...
- benibela 4y agoRecently, I looked at some string functions in the Rust stdlib and noticed it is full of unsafe code. With comments like "this writes ascii only, so we do not utf-8 validation" hat validation is slowing down everything
- fuzzygroup 4y agoDon't things just take forever. I watched how long Ruby took to deal with Unicode. Sigh. But at least its happening.
- zbraniecki 4y agoHi, just for context - this was a comparison of `unicode-normalizer` crate to ICU4C. Since then @hsivonen from Mozilla wrote a new normalizer that recently got merged into ICU4X - https://github.com/unicode-org/icu4x/tree/main/components/normalizer https://github.com/unicode-org/icu4x/tree/main/components/no... I don't have perf numbers yet but I suspect it to be perf comparable to ICU4C at least.