3 ms·
Right, somebody else might have actual metrics but I'd have guessed actual regular expression patterns are split something like: 90% Only care about ASCII, thu
by tialaramex 4y ago
Right, somebody else might have actual metrics but I'd have guessed actual regular expression patterns are split something like:
90% Only care about ASCII, thus individual bytes in UTF-8, and so UTF-32 just wastes memory
1% Care about individual code points, but spread over multiple bytes (e.g. the double dagger ‡), UTF-32 is perfect
9% Care about multiple code points (to form e.g. a Flag, or é written in combining form, or two women kissing) and so UTF-32 doesn't really help again
- bmn__ 4y agoApplications and text-processing libraries are free use an internal text encoding or data structure, if you will, which does not suffer from the drawbacks. Conversions to encodings suitable for data exchange (e.g. UTF-8) are performed at the I/O boundaries.
- tialaramex 4y agoSure, and if stuff lives inside your "application and text-processing library" for long enough and performs enough of the thusly optimised actions without leaving, you might even amortize the cost of the work you did at the edges. But probably not.