5 ms·

How do you detect if a byte swap is needed? I.e. wether the (fixed) wire endianness matches the current platform endianness?

edflsafoiewq 5y ago
Ie how do you know the target's endianness? C++20 added std::endian. Otherwise you can use a macro like this one from SDL https://github.com/libsdl-org/SDL/blob/9dc97afa7190aca5bdf923ed13d723197c083547/include/SDL_endian.h#L61-L82 https://github.com/libsdl-org/SDL/blob/9dc97afa7190aca5bdf92...
- hermitdev 5y ago
  There have been CPU architectures where the endianness at compile time isn't necessarily sufficient. I forget which, maybe it was DEC Alpha, where the CPU could flip back and forth? I can't recall if it was a "choose at boot" or a per process change.
  - magicalhippo 5y ago
    ARM allows dynamic changing of endianess[1]. [1]: https://developer.arm.com/documentation/dui0489/h/arm-and-thumb-instructions/setend https://developer.arm.com/documentation/dui0489/h/arm-and-th...
    - account42 5y ago
      Which nothing will be able to deal with so you might as well not bother to support it. Your compiler will also assume a fixed endianness based on the target triple.
klodolph 5y ago
Using the preprocessor, something like this: uint32_t swap32(uint32_t x) { ... } #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ uint32_t swap32be(uint32_t x) { return swap32(x); } #elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ uint32_t swap32be(uint32_t x) { return x; } #else #error "Unknown endian" #endif You can make the preprocessor condition broader if you care about more compilers and more platforms. Yes, I'm making assumptions about which platforms you want to target... which is fine. No, I don't care about your PDP-11, nor about dynamically changing your endian at runtime. Nearly any problem in C can be made arbitrarily difficult if you care about sufficiently bizarre platforms, or ask that people write code that is correct on any theoretical conforming C implementation. So we pick some platforms to support. The above code is fairly simple. You can separate the part where you care about unaligned memory access and the part where you care about endian. Some irrelevant details left out above.
- jart 5y ago
  Author here. The blog post has that as the naive example. The whole intention was to help people understand why we don't need to do that. Could you at least explain why you disagree if you're going to use this thread to provide the complete opposite advice?
  - klodolph 5y ago
    When I read the blog post I saw this, #define READ32BE(p) bswap_32(*(uint32_t *)(p)) Which as you correctly state in the article, is incorrect code. We agree about this. I proposed an alternate solution, where the READ32BE would be like this: uint32_t read32be(const void *ptr) { uint32_t x; memcpy(&x, ptr, sizeof(x)); return swap32be(x); // Nop on big-endian. } What I like about this is that it breaks the problem down into two parts: reading unaligned data and converting byte order. The reason for this is, sometimes, you need a half of that. Some wire formats have alignment guarantees, and if you know that the alignment guarantees are compatible with your platform, you can just read the data into a buffer and then (optionally) swap the bytes in place. Just to give an example... not too long ago I was working with legacy code that was written for MIPS. Unaligned access does not work on MIPS, so the code was already carefully written to avoid that. All I had to do was make sure that the data types were sized (e.g. replace "long" with "int32_t") and then go through and byte swap everything. struct Something { int32_t x, y; char name[16]; }; void Something_Swap(struct Something *p) { p->x = swap32be(p->x); p->y = swap32be(p->y); } So it's nice to have a function like swap32be(), and "you don't have to mask and shift" I would say is true, it just depends on which compilers you want to support. I would say that a key part of being a C programmer is making a conscious decision about which compilers you want to support. Yes, I'm aware that structs are not a great way to serialize data in general, but sometimes they're damn convenient.

edflsafoiewq 5y ago

Ie how do you know the target's endianness? C++20 added std::endian. Otherwise you can use a macro like this one from SDL https://github.com/libsdl-org/SDL/blob/9dc97afa7190aca5bdf923ed13d723197c083547/include/SDL_endian.h#L61-L82 https://github.com/libsdl-org/SDL/blob/9dc97afa7190aca5bdf92...

hermitdev 5y ago
There have been CPU architectures where the endianness at compile time isn't necessarily sufficient. I forget which, maybe it was DEC Alpha, where the CPU could flip back and forth? I can't recall if it was a "choose at boot" or a per process change.
- magicalhippo 5y ago
  ARM allows dynamic changing of endianess[1]. [1]: https://developer.arm.com/documentation/dui0489/h/arm-and-thumb-instructions/setend https://developer.arm.com/documentation/dui0489/h/arm-and-th...
  - account42 5y ago
    Which nothing will be able to deal with so you might as well not bother to support it. Your compiler will also assume a fixed endianness based on the target triple.

klodolph 5y ago

Using the preprocessor, something like this: uint32_t swap32(uint32_t x) { ... } #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ uint32_t swap32be(uint32_t x) { return swap32(x); } #elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ uint32_t swap32be(uint32_t x) { return x; } #else #error "Unknown endian" #endif You can make the preprocessor condition broader if you care about more compilers and more platforms. Yes, I'm making assumptions about which platforms you want to target... which is fine. No, I don't care about your PDP-11, nor about dynamically changing your endian at runtime. Nearly any problem in C can be made arbitrarily difficult if you care about sufficiently bizarre platforms, or ask that people write code that is correct on any theoretical conforming C implementation. So we pick some platforms to support. The above code is fairly simple. You can separate the part where you care about unaligned memory access and the part where you care about endian. Some irrelevant details left out above.

jart 5y ago
Author here. The blog post has that as the naive example. The whole intention was to help people understand why we don't need to do that. Could you at least explain why you disagree if you're going to use this thread to provide the complete opposite advice?
- klodolph 5y ago
  When I read the blog post I saw this, #define READ32BE(p) bswap_32(*(uint32_t *)(p)) Which as you correctly state in the article, is incorrect code. We agree about this. I proposed an alternate solution, where the READ32BE would be like this: uint32_t read32be(const void *ptr) { uint32_t x; memcpy(&x, ptr, sizeof(x)); return swap32be(x); // Nop on big-endian. } What I like about this is that it breaks the problem down into two parts: reading unaligned data and converting byte order. The reason for this is, sometimes, you need a half of that. Some wire formats have alignment guarantees, and if you know that the alignment guarantees are compatible with your platform, you can just read the data into a buffer and then (optionally) swap the bytes in place. Just to give an example... not too long ago I was working with legacy code that was written for MIPS. Unaligned access does not work on MIPS, so the code was already carefully written to avoid that. All I had to do was make sure that the data types were sized (e.g. replace "long" with "int32_t") and then go through and byte swap everything. struct Something { int32_t x, y; char name[16]; }; void Something_Swap(struct Something *p) { p->x = swap32be(p->x); p->y = swap32be(p->y); } So it's nice to have a function like swap32be(), and "you don't have to mask and shift" I would say is true, it just depends on which compilers you want to support. I would say that a key part of being a C programmer is making a conscious decision about which compilers you want to support. Yes, I'm aware that structs are not a great way to serialize data in general, but sometimes they're damn convenient.