4 ms·
While the backward compatibility of utf-8 is nice, and makes adoption much easier, the backward compatibility does not come at any cost to the elegance of the e
by bruce511 1y ago
While the backward compatibility of utf-8 is nice, and makes adoption much easier, the backward compatibility does not come at any cost to the elegance of the encoding.
In other words, yes it's backward compatible, but utf-is also compact and elegant even without that.
- nextaccountic 1y agoUTF-8 also enables this mindblowing design for small string optimization - if the string has 24 bytes or less it is stored inline, otherwise it is stored on the heap (with a pointer, a length, and a capacity - also 24 bytes) https://github.com/ParkMyCar/compact_str https://github.com/ParkMyCar/compact_str How cool is that (Discussed here https://news.ycombinator.com/item?id=41339224 https://news.ycombinator.com/item?id=41339224)
- adgjlsfhk1 1y agoHow is that UTF8 specific?
- ubitaco 1y agoIt's slightly buried in the readme on Github: > how can we store a 24 byte long string, inline? Don't we also need to store the length somewhere? > To do this, we utilize the fact that the last byte of our string could only ever have a value in the range [0, 192). We know this because all strings in Rust are valid UTF-8, and the only valid byte pattern for the last byte of a UTF-8 character (and thus the possible last byte of a string) is 0b0XXXXXXX aka [0, 128) or 0b10XXXXXX aka [128, 192)
- Dylan16807 1y agoAny Unicode encoding would allow that. UTF-32 has an entire spare byte to put flags into. 24 or 21 bit encodings have spare bits that could act as flags. UTF-16 has plenty of invalid code units, or you could use a high surrogate in the last 2 bytes as your flag.
- deleted 1y ago[deleted]