7 ms·
I'm an idiot in this area. Why would you decode utf8?
by kuwze 9y ago
I'm an idiot in this area.
Why would you decode utf8?
- fra 9y agoIn order to look up glyph in your font, you'll typically need a 32bit unicode code-point. A utf-8 string encodes those code-points using a variable-number-of-bytes scheme, so you need to decode each code-point before you can render it.
- amelius 9y agoYes, but this code expands it to a 32-bit array. That seems a bit silly because you'd be wasting a lot of memory bandwidth there. I suspect the loss could even be greater than what you've gained from removing those branches.
- amaranth 9y agoNot quite, it's expanded to a 32-bit integer, one codepoint at a time.
- grzm 9y agoPerhaps you're misinterpreting what a decoder does. From the article: > This week I took a crack at writing a branchless UTF-8 decoder: a function that decodes a single UTF-8 code point from a byte stream…
- chrisseaton 9y agoTo print it to the screen for example.
- smegel 9y agoSome languages represent unicode strings as objects quite separate to their binary encoding, e.g. Python.
- kutkloon7 9y agoTo get the unicode codepoint. For example, you need this when you want to render a character, or convert it to another representation.