4 ms·
It basically has to be distinct from `char` because you can't use portably use `char` to hold a UTF-8 code unit (because the guaranteed valid range is only 0x00
by Sharlin 5y ago
It basically has to be distinct from `char` because you can't use portably use `char` to hold a UTF-8 code unit (because the guaranteed valid range is only 0x00 to 0x7F) Also, this way you can overload based on legacy char vs UTF-8 char, and have `std::basic_string<char>` and `std::basic_string<char8_t>` (aka `std::string` and std::u8string`) be distinct types as well. So finally in C++20 we actually have a portable UTF-8 string type!
- ncmncm 5y agoAnyway, a UTF-8 sequence transport type. Few of u8string member operations make sense for UTF-8 as such. Usually that doesn't matter. Sometimes it matters a great deal, and we will need a whole new API for that.
- Sharlin 5y agoYeah, good point.
- planede 5y ago> you can't use portably use `char` to hold a UTF-8 code unit That's not true, in C++ a byte is guaranteed to be able to hold a UTF-8 code unit. https://timsong-cpp.github.io/cppwp/n4868/intro.memory#1.sentence-2 https://timsong-cpp.github.io/cppwp/n4868/intro.memory#1.sen...
- Sharlin 5y agoYes, but if `char` is signed, as it usually is, its bit patterns correspond to values -0x80 to 0x7F. So yeah, you can no-cost encode the >=0x80 code units as their two’s complement counterparts but it feels suspicious. At least to me, after writing some Rust lately which very much does not do implicit signed–unsigned conversions. Much better for char to always represent the "basic character set" (ie. usually ASCII) and have a distinct type for UTF-8.