4 ms·

There are a lot of errors in this article and the code therein. i = ((int)data); #ifdef BIG_ENDIAN /* swap the bytes */ i = ((i&0xFF)<<24) | (((i>>8)

There are a lot of errors in this article and the code therein. i = *((int*)data); #ifdef BIG_ENDIAN /* swap the bytes */ i = ((i&0xFF)<<24) | (((i>>8)&0xFF)<<16) | (((i>>16)&0xFF)<<8) | (((i>>24)&0xFF)<<0); #endif This should use uint32_t, it is the best way of getting a platform independent unsigned 32-bit integer, which is what you want here. It's more code. Couple more lines of C, yes. No more at the machine level. It assumes integers are addressible at any byte offset; on some machines that's not true. Not sure about this one... It depends on integers being 32 bits long, or requires more #ifdefs to pick a 32-bit integer type. This is caused by bad code - see above. It may be a little faster on little-endian machines, but not much, and it's slower on big-endian machines. It is faster on a LE machine, but not slower on a BE machine - the same code can be used and it's a compile time #ifdef. If you're using a little-endian machine when you write this, there's no way to test the big-endian code. Test on a BE machine? It swaps the bytes, a sure sign of trouble (see below). No actual facts here... As pointed out by another commentor, this can be optimised out by the compiler on many platforms.

paulsutter 15y ago
While the article is rant-y and some details sloppy, I have to admit his core point is pretty good. Rather the writing #ifdef'd code, why not write code that works the same regardless of endianness? That seems like a good idea. Not for -all- the reasons he mentions, but for the simple reason that it's one code path so easier to code and easier to get right.
- alexchamberlain 15y ago
  Started a github repo at https://github.com/alexchamberlain/byte-order https://github.com/alexchamberlain/byte-order.
  - duaneb 15y ago
    If you're using C++, you can dispose of the ifdefs completely and check at compile time the union of a uint16_t and two uint8_ts.
    - alexchamberlain 15y ago
      The C++ is just an interface... I'm just nice like that!
  - deleted 15y ago
    [deleted]
dfox 15y ago
The point is, when you use explicit shifting of bytes you have same code that works independently of endianity, the fact that compiler is probably going to generate exactly same code seems to me like good argument to go with the more readable choice (ie. explicit shifts), also variant proposed by article is actually portable C, anything involving casting arrays of one type to arrays of another incompatible type is not. No modern architecture can access arbitrarily aligned words in memory directly (presence of caches modifies things slightly, but shifts the problem from data bus width to cache line width as unaligned word can still span two cache lines). There are generally two solutions to this: disallow that at CPU level (and handle that by raising SIGBUS), emulate it in hardware by doing two memory accesses for one load or store (which involves significant additional complexity), Intel invented third solution in i386: OS can select between these two behaviors.
- alexchamberlain 15y ago
  I would argue we, as a community, need to write a portable, yet optimised, byte order convertors. htobe, htole, htobel, htolel, htobell, htolell, and vice versa.
  - chronomex 15y ago
    Please see the "htonl" function and its brethren in <arpa/inet.h>.
    - alexchamberlain 15y ago
      As stated elsewhere, these do host to big endian and vice versa. No little endian support, though most of us are working on LE machines ofc.
  - deleted 15y ago
    [deleted]
  - dfox 15y ago
    Converting byte order of integer is mostly pointless operation (which is what the article tries to say), what is needed is portable, yet optimized way to build/parse portable binary structures. In my opinion there are two reasons why too much optimization in this is complete waste of time: 1) Even if compilers are not able to optimize manual conversion of integer to/from discrete bytes into same code as word sized access with optional byte order swap, it's mostly irrelevant, as there aren't going to be any significant difference in performance between one four byte access and four one byte accesses (as in both cases you end up with same number of actual memory transactions, which is the slow part, due to caches) 2) when you are handling portable binary representation of something, it's always connected to some IO, which is slow already so any performance boost that you get from microoptimalization like this is completely negligible. I tend to just hand write few lines of C to pack/unpack integers explicitly when needed as it seems to me as the most productive thing you can do. By the way all the big endian <-> little endian functions you propose boil down to two implementations for each size of operand: no-op and mirroring of all bytes, both of which are mostly trivial. What is really missing is portable and efficient way to encode floating point numbers, as there is no portable way to find out their endianity and in floating point case it's more complex than just big vs. little endian.

There are a lot of errors in this article and the code therein. i = *((int*)data); #ifdef BIG_ENDIAN /* swap the bytes */ i = ((i&0xFF)<<24) | (((i>>8)

paulsutter 15y ago

While the article is rant-y and some details sloppy, I have to admit his core point is pretty good. Rather the writing #ifdef'd code, why not write code that works the same regardless of endianness? That seems like a good idea. Not for -all- the reasons he mentions, but for the simple reason that it's one code path so easier to code and easier to get right.

alexchamberlain 15y ago
Started a github repo at https://github.com/alexchamberlain/byte-order https://github.com/alexchamberlain/byte-order.
- duaneb 15y ago
  If you're using C++, you can dispose of the ifdefs completely and check at compile time the union of a uint16_t and two uint8_ts.
  - alexchamberlain 15y ago
    The C++ is just an interface... I'm just nice like that!
- deleted 15y ago
  [deleted]

dfox 15y ago

The point is, when you use explicit shifting of bytes you have same code that works independently of endianity, the fact that compiler is probably going to generate exactly same code seems to me like good argument to go with the more readable choice (ie. explicit shifts), also variant proposed by article is actually portable C, anything involving casting arrays of one type to arrays of another incompatible type is not. No modern architecture can access arbitrarily aligned words in memory directly (presence of caches modifies things slightly, but shifts the problem from data bus width to cache line width as unaligned word can still span two cache lines). There are generally two solutions to this: disallow that at CPU level (and handle that by raising SIGBUS), emulate it in hardware by doing two memory accesses for one load or store (which involves significant additional complexity), Intel invented third solution in i386: OS can select between these two behaviors.

alexchamberlain 15y ago
I would argue we, as a community, need to write a portable, yet optimised, byte order convertors. htobe, htole, htobel, htolel, htobell, htolell, and vice versa.
- chronomex 15y ago
  Please see the "htonl" function and its brethren in <arpa/inet.h>.
  - alexchamberlain 15y ago
    As stated elsewhere, these do host to big endian and vice versa. No little endian support, though most of us are working on LE machines ofc.
- deleted 15y ago
  [deleted]
- dfox 15y ago
  Converting byte order of integer is mostly pointless operation (which is what the article tries to say), what is needed is portable, yet optimized way to build/parse portable binary structures. In my opinion there are two reasons why too much optimization in this is complete waste of time: 1) Even if compilers are not able to optimize manual conversion of integer to/from discrete bytes into same code as word sized access with optional byte order swap, it's mostly irrelevant, as there aren't going to be any significant difference in performance between one four byte access and four one byte accesses (as in both cases you end up with same number of actual memory transactions, which is the slow part, due to caches) 2) when you are handling portable binary representation of something, it's always connected to some IO, which is slow already so any performance boost that you get from microoptimalization like this is completely negligible. I tend to just hand write few lines of C to pack/unpack integers explicitly when needed as it seems to me as the most productive thing you can do. By the way all the big endian <-> little endian functions you propose boil down to two implementations for each size of operand: no-op and mirroring of all bytes, both of which are mostly trivial. What is really missing is portable and efficient way to encode floating point numbers, as there is no portable way to find out their endianity and in floating point case it's more complex than just big vs. little endian.