4 ms·

The surprising thing to me is that 'char8_t' (a new C++ 20 thing I'd never heard of) is not just a typedef to char with all the same aliasing implications, but

The surprising thing to me is that 'char8_t' (a new C++ 20 thing I'd never heard of) is not just a typedef to char with all the same aliasing implications, but a new and distinct type to which the magic 'char' alias rules don't apply (also, unsigned).

matheusmoreira 4y ago
I agree, it's surprising. Types like uint8_t* are widely used to reinterpret other structures as byte arrays which implies they are universal aliases just like char*. Not sure what makes char8_t different.
- wahern 4y ago
  > Types like uint8_t* are widely used to reinterpret other structures as byte arrays which implies they are universal aliases just like char* Alternatively, it implies that there's alot of broken code out there. So much broken code that they've accidentally found safety in numbers, and compilers are unlikely to change a coincidental behavior upon which they wrongly relied. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66110 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66110
MauranKilom 4y ago
Also, char is a distinct type from both signed char and unsigned char (even though it has the same size as both and the same signedness as one of them).
- jhgb 4y ago
  Is signed char and unsigned char subject to the same "can alias anything" rule? I never bothered to think about this in the past. Now I'm not sure. I've known that the three types are distinct, but that just means that that the rule doesn't have to apply to all of them, not that it doesn't.
  - saagarjha 4y ago
    Yes, the rule applies to all three character types.
    - jhgb 4y ago
      Thanks. This seems mildly ungoogleable, or at least I haven't been able to find a good search result for this. So I'll keep in mind that these are three distinct types with the same aliasing behavior.
      - saagarjha 4y ago
        Actually, I take that back and should clarify: it's all three only in C. In C++ it is just char and unsigned char. (If you want to search for this, "signed char aliasing" gave me good results.)
- quibono 4y ago
  Is there any difference in using char vs unsigned char then? I understand these two types are different as far as the compiler's concerned. Would they behave differently too?
  - MauranKilom 4y ago
    signedness of char is up to the implementation. You should not rely on char behaving like unsigned char.
Sharlin 4y ago
It basically has to be distinct from `char` because you can't use portably use `char` to hold a UTF-8 code unit (because the guaranteed valid range is only 0x00 to 0x7F) Also, this way you can overload based on legacy char vs UTF-8 char, and have `std::basic_string<char>` and `std::basic_string<char8_t>` (aka `std::string` and std::u8string`) be distinct types as well. So finally in C++20 we actually have a portable UTF-8 string type!
- ncmncm 4y ago
  Anyway, a UTF-8 sequence transport type. Few of u8string member operations make sense for UTF-8 as such. Usually that doesn't matter. Sometimes it matters a great deal, and we will need a whole new API for that.
  - Sharlin 4y ago
    Yeah, good point.
- planede 4y ago
  > you can't use portably use `char` to hold a UTF-8 code unit That's not true, in C++ a byte is guaranteed to be able to hold a UTF-8 code unit. https://timsong-cpp.github.io/cppwp/n4868/intro.memory#1.sentence-2 https://timsong-cpp.github.io/cppwp/n4868/intro.memory#1.sen...
  - Sharlin 4y ago
    Yes, but if `char` is signed, as it usually is, its bit patterns correspond to values -0x80 to 0x7F. So yeah, you can no-cost encode the >=0x80 code units as their two’s complement counterparts but it feels suspicious. At least to me, after writing some Rust lately which very much does not do implicit signed–unsigned conversions. Much better for char to always represent the "basic character set" (ie. usually ASCII) and have a distinct type for UTF-8.
ncmncm 4y ago
And also distinct from std::byte, which they are hoping will pick up aliasing properties of char, allowing use and also abuse of char* to be someday eliminated from new code. std::byte does alias everything, but no operations are defined on it except copying (and, weirdly, bitwise operators).

matheusmoreira 4y ago

I agree, it's surprising. Types like uint8_t* are widely used to reinterpret other structures as byte arrays which implies they are universal aliases just like char*. Not sure what makes char8_t different.

wahern 4y ago
> Types like uint8_t* are widely used to reinterpret other structures as byte arrays which implies they are universal aliases just like char* Alternatively, it implies that there's alot of broken code out there. So much broken code that they've accidentally found safety in numbers, and compilers are unlikely to change a coincidental behavior upon which they wrongly relied. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66110 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66110

MauranKilom 4y ago

Also, char is a distinct type from both signed char and unsigned char (even though it has the same size as both and the same signedness as one of them).

jhgb 4y ago
Is signed char and unsigned char subject to the same "can alias anything" rule? I never bothered to think about this in the past. Now I'm not sure. I've known that the three types are distinct, but that just means that that the rule doesn't have to apply to all of them, not that it doesn't.
- saagarjha 4y ago
  Yes, the rule applies to all three character types.
  - jhgb 4y ago
    Thanks. This seems mildly ungoogleable, or at least I haven't been able to find a good search result for this. So I'll keep in mind that these are three distinct types with the same aliasing behavior.
    - saagarjha 4y ago
      Actually, I take that back and should clarify: it's all three only in C. In C++ it is just char and unsigned char. (If you want to search for this, "signed char aliasing" gave me good results.)
quibono 4y ago
Is there any difference in using char vs unsigned char then? I understand these two types are different as far as the compiler's concerned. Would they behave differently too?
- MauranKilom 4y ago
  signedness of char is up to the implementation. You should not rely on char behaving like unsigned char.

Sharlin 4y ago

It basically has to be distinct from `char` because you can't use portably use `char` to hold a UTF-8 code unit (because the guaranteed valid range is only 0x00 to 0x7F) Also, this way you can overload based on legacy char vs UTF-8 char, and have `std::basic_string<char>` and `std::basic_string<char8_t>` (aka `std::string` and std::u8string`) be distinct types as well. So finally in C++20 we actually have a portable UTF-8 string type!

ncmncm 4y ago
Anyway, a UTF-8 sequence transport type. Few of u8string member operations make sense for UTF-8 as such. Usually that doesn't matter. Sometimes it matters a great deal, and we will need a whole new API for that.
- Sharlin 4y ago
  Yeah, good point.
planede 4y ago
> you can't use portably use `char` to hold a UTF-8 code unit That's not true, in C++ a byte is guaranteed to be able to hold a UTF-8 code unit. https://timsong-cpp.github.io/cppwp/n4868/intro.memory#1.sentence-2 https://timsong-cpp.github.io/cppwp/n4868/intro.memory#1.sen...
- Sharlin 4y ago
  Yes, but if `char` is signed, as it usually is, its bit patterns correspond to values -0x80 to 0x7F. So yeah, you can no-cost encode the >=0x80 code units as their two’s complement counterparts but it feels suspicious. At least to me, after writing some Rust lately which very much does not do implicit signed–unsigned conversions. Much better for char to always represent the "basic character set" (ie. usually ASCII) and have a distinct type for UTF-8.

ncmncm 4y ago

And also distinct from std::byte, which they are hoping will pick up aliasing properties of char, allowing use and also abuse of char* to be someday eliminated from new code. std::byte does alias everything, but no operations are defined on it except copying (and, weirdly, bitwise operators).