2 ms·
Thank you. I was looking for something that takes a potentially invalid buffer of UTF-8 and returns a guaranteed-valid buffer and failed to find a function like
by hsivonen 9y ago
Thank you. I was looking for something that takes a potentially invalid buffer of UTF-8 and returns a guaranteed-valid buffer and failed to find a function like that.
(And, indeed, Go is an interesting case due to its creators being the inventors of UTF-8, too.)
- nigeltao 9y agoYeah, there's not really a guaranteed-valid buffer concept in Go. Even if you have valid UTF-8, you still have to iterate over it to e.g. rasterize glyphs, and iterating over possibly-bad UTF-8 is no harder than iterating over known-good UTF-8. If you want to compare to other UTF-8, validity alone isn't always sufficient. You often have to e.g. normalize anyway, and normalization should fix up bad UTF-8. Again, a guaranteed-valid buffer type wouldn't win you much.
- nigeltao 9y agoOh, in case you were wondering, the ubiquitous term "rune" in the Go documentation is simply shorthand for "Unicode codepoint".