3 ms·

Especially in data structures if you are hitting the disk or network - the data structure alignment can get screwed up once it hits another machine. The only i

Especially in data structures if you are hitting the disk or network - the data structure alignment can get screwed up once it hits another machine. The only integer type you should be using in data structures which are going out to disk or network is uint8_t. Otherwise the uint32_t you wrote to disk on an x86 system might end up getting read on a PDP-11 and suddenly your little-endian value is middle-endian instead. (I would love to see the C standard grow explicitly-endian types so you could write e.g. "ubeint32_t x = 0x12345678;" and have the value serialized in big-endian form. But alas the C standard moves slowly.)

kristopolous 3y ago
In practice I just hton[sl] it for both - it's the great equalizer and will get optimized out when not needed at even modest -O levels. If I'm going to crack open an int32, it's htonl all day long and ntohl it back when I'm done.
chongli 3y ago
What does it mean to have an "explicitly-endian" type? In the register, it's all the same thing. I have seen tons of code with #if BIG_ENDIAN ... and it's a huge nightmare. Follow the Rob Pike way [1] and regain sanity. [1] https://commandcenter.blogspot.com/2012/04/byte-order-fallacy.html https://commandcenter.blogspot.com/2012/04/byte-order-fallac...
- corysama 3y ago
  Not just sane, but fast too. Compilers have long been able to recognize this algo and will optimize all those bit ops to a single BSWAP assembly instruction as necessary.
- cperciva 3y ago
  What does it mean to have an "explicitly-endian" type? In the register, it's all the same thing. An integer type with an explicit representation in memory. In this case, "arithmetic works like a uint32_t, but when you're storing it to memory, do it in big-endian format". Follow the Rob Pike way [1] and regain sanity. This is the Rob Pike way, except it would involve the compiler writing the code for converting between uint32_t and uint8_t[4].
Dylan16807 3y ago
> (I would love to see the C standard grow explicitly-endian types so you could write e.g. "ubeint32_t x = 0x12345678;" and have the value serialized in big-endian form. But alas the C standard moves slowly.) How would that work? If it keeps that layout all the time performance will be terrible, and if you want to convert it when serializing how do you intend to trigger that conversion? (And is that trigger something you couldn't do with a library?)
- cperciva 3y ago
  For in-memory operations you would use uint32_t as normal, but for the memory structures which are going to be written to disk or sent over the network you would have ubeint32_t. When you assign the value from one to the other, the compiler would insert a byteswap if necessary. Basically this would be a simpler replacement for explicit serialization of integers into bytes.
  - Dylan16807 3y ago
    So I have to write two versions of the data structure? I don't know if that sounds very simple to me, but it does sound like you could do that with a library and no language changes.
    - cperciva 3y ago
      You probably have two versions of the data structure anyway, since the data structure you want in memory is unlikely to be the data structure you want on disk or over the network. Your in-memory data structure probably has pointers, for example.
jltsiren 3y ago
I prefer doing some quick sanity checks and then printing an error message like "User error: Your computer is weird. Buy a normal computer." if they fail. The convenience of being able to use memory-mapped files directly without any endianness / alignment bureaucracy is more important to me than supporting some hypothetical hardware I don't have access to. Low-level code like that rarely lives beyond 5-10 years anyway. Even if some weird future hardware starts getting popular, the code will most likely be rewritten at least once before that hardware actually matters.

kristopolous 3y ago

In practice I just hton[sl] it for both - it's the great equalizer and will get optimized out when not needed at even modest -O levels. If I'm going to crack open an int32, it's htonl all day long and ntohl it back when I'm done.

chongli 3y ago

What does it mean to have an "explicitly-endian" type? In the register, it's all the same thing. I have seen tons of code with #if BIG_ENDIAN ... and it's a huge nightmare. Follow the Rob Pike way [1] and regain sanity. [1] https://commandcenter.blogspot.com/2012/04/byte-order-fallacy.html https://commandcenter.blogspot.com/2012/04/byte-order-fallac...

corysama 3y ago
Not just sane, but fast too. Compilers have long been able to recognize this algo and will optimize all those bit ops to a single BSWAP assembly instruction as necessary.
cperciva 3y ago
What does it mean to have an "explicitly-endian" type? In the register, it's all the same thing. An integer type with an explicit representation in memory. In this case, "arithmetic works like a uint32_t, but when you're storing it to memory, do it in big-endian format". Follow the Rob Pike way [1] and regain sanity. This is the Rob Pike way, except it would involve the compiler writing the code for converting between uint32_t and uint8_t[4].

Dylan16807 3y ago

> (I would love to see the C standard grow explicitly-endian types so you could write e.g. "ubeint32_t x = 0x12345678;" and have the value serialized in big-endian form. But alas the C standard moves slowly.) How would that work? If it keeps that layout all the time performance will be terrible, and if you want to convert it when serializing how do you intend to trigger that conversion? (And is that trigger something you couldn't do with a library?)

cperciva 3y ago
For in-memory operations you would use uint32_t as normal, but for the memory structures which are going to be written to disk or sent over the network you would have ubeint32_t. When you assign the value from one to the other, the compiler would insert a byteswap if necessary. Basically this would be a simpler replacement for explicit serialization of integers into bytes.
- Dylan16807 3y ago
  So I have to write two versions of the data structure? I don't know if that sounds very simple to me, but it does sound like you could do that with a library and no language changes.
  - cperciva 3y ago
    You probably have two versions of the data structure anyway, since the data structure you want in memory is unlikely to be the data structure you want on disk or over the network. Your in-memory data structure probably has pointers, for example.

jltsiren 3y ago

I prefer doing some quick sanity checks and then printing an error message like "User error: Your computer is weird. Buy a normal computer." if they fail. The convenience of being able to use memory-mapped files directly without any endianness / alignment bureaucracy is more important to me than supporting some hypothetical hardware I don't have access to. Low-level code like that rarely lives beyond 5-10 years anyway. Even if some weird future hardware starts getting popular, the code will most likely be rewritten at least once before that hardware actually matters.