3 ms·
If the natural logic is broken then you've misunderstood the thing's nature. UTF-8 behaves more like a communications stream that was shoehorned into the purpos
by jart 6y ago
If the natural logic is broken then you've misunderstood the thing's nature. UTF-8 behaves more like a communications stream that was shoehorned into the purpose of character arrays. If you think about it in that way, then decoding can be done simply and elegantly:
#define bsr(u) ((sizeof(int) * 8 - 1) ^ __builtin_clz(u))
#define ThomPikeByte(x) ((x) & (((1 << ThomPikeMsb(x)) - 1) | 3))
#define ThomPikeMsb(x) (((x)&0xff) < 252 ? bsr(~(x)&0xff) : 1)
void ThompsonPikeDecoder(const char *s) {
unsigned c, w = 0, t = 0;
do {
c = *s++ & 0xff;
if (0200 <= c && c < 0300) {
w = w << 6 | c & 077;
} else {
if (t) {
printf("%04x\n", w);
t = 0;
}
if (c < 0200) {
printf("%04x\n", c);
} else {
w = ThomPikeByte(c);
t = 1;
}
}
} while (c);
}
That code generalizes to any 32-bit number. It's the full superset of intended behaviors including things like stream synchronization. It'll even decode numbers that were arbitrarily banned by the IETF e.g. \300\200. Most importantly, since it doesn't require a validation pass beforehand, does that mean it goes faster than the OP's code in praxis? D:
- Dylan16807 6y agoI strongly question "doesn't require a validation pass". 1. Your code drops naked continuation bytes entirely. 2. If a valid multibyte character is followed by extraneous continuation bytes it will not decode correctly. 3. Using the wrong number of continuation bytes is a giant mess and can give you arbitrary numbers in multiple ways. The second one especially is a behavior that's hard to defend.