8 ms·

The long term good news is that the endian wars are drawing to a close. Little Endian won. I believe all the most recent CPU's support little endian at least as

The long term good news is that the endian wars are drawing to a close. Little Endian won. I believe all the most recent CPU's support little endian at least as an option. We will still have to deal with the legacy network protocols that have big endian encoding. By the way, if you are designing a new network binary protocol, please do not pick big endian, it just makes stuff more complicated on the majority of current cpus.

plq 6y ago
I wonder whether that's due to any technical merit of little endianness or just because of x86's dominance in the desktop/workstation world?
- moonchild 6y ago
  Little endian is better. Most arithmetic needs to start with the less significant bits, so it's better to have them be accessible first. (E.G. most bignum representations also use little-endian, even though they don't have to.) Little endian also allows you to truncate integers more easily.
  - codesnik 6y ago
    but it's really weird to look on those in hexdump
    - Artlav 6y ago
      And that's another reason - gone are the days when code was written in machine language by people looking at bytes, and with it gone the only reason to keep these bytes in human-readable form. It's the same kind of thing as computers and calculators switching from decimal hardware to binary hardware 100 years ago.
      - twic 6y ago
        People still routinely look at data in hexdumps, and little-endian is still as irritating as it ever was.
      - DerekL 6y ago
        > It's the same kind of thing as computers and calculators switching from decimal hardware to binary hardware 100 years ago. Not quite. ENIAC, for example, did calculations in decimal, and it was finished in 1945. There may be more recent examples. If you include binary-coded decimal, that still includes machines made in 1970 or later, like the IBM 1401. https://en.wikipedia.org/wiki/IBM_1401 https://en.wikipedia.org/wiki/IBM_1401 https://news.ycombinator.com/item?id=25523904 https://news.ycombinator.com/item?id=25523904
  - viraptor 6y ago
    Does it actually matter these days? Registers can be wired in any way physically, so there's no "first byte" on that level. It would only matter if there was an instruction pipeline optimising an 8bit bus reading the long value into ALU input and ALU doing the operation... which as far as I understand modern design doesn't actually happen.
    - adrian_b 6y ago
      The parent meant that it matters for multi-word numbers, where the endianness becomes visible. In the days of 8-bit microprocessors, the endianness became visible in software from 16-bit numbers up. Now it might be visible only for numbers above 64-bit on many CPUs, but there is always a number size above which you must take endianness into account, even for the simplest operations, like comparing and adding. In theory, you could choose the layout of a large number to have mixed endianness, e.g. to have a large number as a little-endian sequence of 64-bit chunks, where the 64-bit chunks are big-endian, because the CPU used is BE. However that might make the algorithms even more complex than when you use a consistent endianness. The endianness of a CPU is not visible in software only when the addressing unit of the memory is the same as the data register size of the CPU. All modern CPUs have register sizes of either 32-bit or 64-bit, while the memory is addressed in bytes, so the endianness will continue to matter in the foreseeable future.
    - moonchild 6y ago
      Registers don't have endianness. Memory does.
  - deleted 6y ago
    [deleted]
  - twic 6y ago
    > Most arithmetic needs to start with the less significant bits, so it's better to have them be accessible first. (E.G. most bignum representations also use little-endian, even though they don't have to.) Some operations start at the least significant end, some start at the most significant end, and it's equally easy to loop over the words in either direction, so i don't buy this. > Little endian also allows you to truncate integers more easily. What do you mean by truncate an integer? If you mean go from 32 bit to 16 bit, that's still trivial with big-endian, you just shift.
    - moonchild 6y ago
      > If you mean go from 32 bit to 16 bit, that's still trivial with big-endian, you just shift. Surely you would agree that moves are easier than shifts? Moves can be handled by the renamer and be effectively free (even for memory operands, on CISC, with the advent of memory renamers; but this is even more important for RISC, where what would otherwise have been a load followed by a shift can just be a load), whereas shifts cannot and must take up some fraction of a cycle.
  - daneel_w 6y ago
    It's a legacy from days of old when it made silicon design of the memory bus easier. Arm, for example, does not incur a performance hit when switching to big-endian.
- jzwinck 6y ago
  Little endian has one nice feature for network communication: integer fields can be expanded or shrunk without changing code of the peer so long as the actual numbers don't overflow. Most mainstream protocols don't really take advantage of this because it is error prone, but it is used sometimes.
  - kaszanka 6y ago
    I don't understand, wouldn't all of the fields after the expanded/shrunk one become misaligned?
    - jzwinck 6y ago
      It only works if you have some way to know where the next field or message starts. Sometimes you do, sometimes you don't.
- gsnedders 6y ago
  A lot of it is simply the prevalence of little-endian systems, initially with x86, but now with the vast majority of other systems too; code is developed on the developer's system, and works there. As big-endian systems have become increasingly rare, an ever larger amount of code has stopped working (or never worked) on big-endian systems, thus it's become harder and harder to ship big-endian systems. If you want desktop-class applications, they've typically either been little-endian only, or have been little-endian only since Apple migrated to Intel. We've seen POWER largely migrate to little-endian, as IBM was struggling with sales due to the difficulty in porting software to it, partly due to the endianness difference. (Note that POWER and PowerPC were technically always bi-endian architectures, but the majority of historic hardware had its endianness set by the motherboard in use, and almost everything shipped with big-endian. Systems running Windows NT on PowerPC were some of the very few that ran in little-endian mode.) We've also seen a similar shift in embedded hardware; bi-endian architectures like ARM and MIPS are now more commonly used in little-endian mode, despite historically being used in big-endian mode.
jzwinck 6y ago
I think the majority of binary integer data sent and received today is big endian. Calling it "legacy" seems like a stretch. People are still designing plenty of brand new protocols using big endian. When I encounter a little endian one I am pleasantly surprised, but it is still a surprise.
- viraptor 6y ago
  I wonder how much of that design is a conscious decision and how much is "we need to use common endianness, stuff's on the network, let's use network-endian"? If we didn't have the label of network byte order, would people default to little?
  - smcameron 6y ago
    Another point is, if you use big endian on the network, and native is little endian, then if you screw up and forget to do the conversion, you catch it because it breaks. If you use little endian for native and network and you screw up, you never catch it unless you run on a big endian machine. It's a way to ensure that you implemented the protocol correctly in an endian clean way. (Of course you could also reverse the sense of the macros during testing, I suppose.)
cbmuser 6y ago
> RcouF1uZ4gsC 26 minutes ago [–] The long term good news is that the endian wars are drawing to a close. Little Endian won. I believe all the most recent CPU's support little endian at least as an option. From what I know from talking to IBM engineers, there are no plans to switch AIX on POWER or Linux on s390x to little-endian. And both are important platforms in the enterprise world.
- viraptor 6y ago
  Based on the number of units of those machines running in production, we can pretty safely treat them as exceptions these days. And people running them can afford to deal with changes needed for big endian conversion.
  - cbmuser 6y ago
    Basically everything that is banks, governments, insurance companies used these machines. They’re not going anywhere.
    - viraptor 6y ago
      They're outnumbered by consumer level little endian devices by 6 orders of magnitude? More? Unless you know you're targeting those industries, you can likely ignore big endian. If they really want your code, you could always charge extra for big endian support too :-P I know they're not going anywhere, but they're also largely irrelevant to majority of us.
BelenusMordred 6y ago
> Little Endian won. If you could tell cryptographers this I would be very grateful.
- cesarb 6y ago
  They already know. Many recent cryptographic algorithms (Curve25519, ChaCha20, and so on) treat their input and output as little-endian numbers.
goalieca 6y ago
Big endian won the encoding wars with many important formats using big endian. Nearly all crypto is big endian for instance because of big number byte strings.
smcameron 6y ago
SCSI also generally uses big endian whenever there are multibyte numbers in the protocol.
Taniwha 6y ago
Strictly speaking no one uses big endian, all computers I know of are either little endian or mixed endian ('mixed. because they do strings low to high but numbers high to low). The only reason that big/mixed endian is even an issue is because some monks fucked up back when the Moors were driven from Spain ... in the wonderful libraries that were left behind they discovered Arabic numbers and realised how useful they were, far better than the clunky Roman ones - for a start a small businessman could do addition just right there on the page, writing the digits from smallest to largest as they went - sadly as I said the monks screwed up and copied the digit order from the right to left written Arabic into western languages without doing the sane thing and reversing the digits so that they continued to be written smallest to largest (so that you could do addition without knowing how much space you needed for digits until you were done). And so we're stuck with this silly artifact of history that has caused decades of arguing as we slowly rediscover the utility of putting the smaller digits first, as they were always meant to be
- nabla9 6y ago
  One of the main causes of the fall of the Roman Empire was that without zero they had no way to indicate successful termination of their C programs.
  - Taniwha 6y ago
    that plus there was no year 0, that surely started a series of events that eventually caused the fall of the Republic
    - RcouF1uZ4gsC 6y ago
      The Republic had already fallen by that time.
- fanf2 6y ago
  Strings are always big endian because the most significant bit comes first when you compare them lexicographically.

plq 6y ago

I wonder whether that's due to any technical merit of little endianness or just because of x86's dominance in the desktop/workstation world?

moonchild 6y ago
Little endian is better. Most arithmetic needs to start with the less significant bits, so it's better to have them be accessible first. (E.G. most bignum representations also use little-endian, even though they don't have to.) Little endian also allows you to truncate integers more easily.
- codesnik 6y ago
  but it's really weird to look on those in hexdump
  - Artlav 6y ago
    And that's another reason - gone are the days when code was written in machine language by people looking at bytes, and with it gone the only reason to keep these bytes in human-readable form. It's the same kind of thing as computers and calculators switching from decimal hardware to binary hardware 100 years ago.
    - twic 6y ago
      People still routinely look at data in hexdumps, and little-endian is still as irritating as it ever was.
    - DerekL 6y ago
      > It's the same kind of thing as computers and calculators switching from decimal hardware to binary hardware 100 years ago. Not quite. ENIAC, for example, did calculations in decimal, and it was finished in 1945. There may be more recent examples. If you include binary-coded decimal, that still includes machines made in 1970 or later, like the IBM 1401. https://en.wikipedia.org/wiki/IBM_1401 https://en.wikipedia.org/wiki/IBM_1401 https://news.ycombinator.com/item?id=25523904 https://news.ycombinator.com/item?id=25523904
- viraptor 6y ago
  Does it actually matter these days? Registers can be wired in any way physically, so there's no "first byte" on that level. It would only matter if there was an instruction pipeline optimising an 8bit bus reading the long value into ALU input and ALU doing the operation... which as far as I understand modern design doesn't actually happen.
  - adrian_b 6y ago
    The parent meant that it matters for multi-word numbers, where the endianness becomes visible. In the days of 8-bit microprocessors, the endianness became visible in software from 16-bit numbers up. Now it might be visible only for numbers above 64-bit on many CPUs, but there is always a number size above which you must take endianness into account, even for the simplest operations, like comparing and adding. In theory, you could choose the layout of a large number to have mixed endianness, e.g. to have a large number as a little-endian sequence of 64-bit chunks, where the 64-bit chunks are big-endian, because the CPU used is BE. However that might make the algorithms even more complex than when you use a consistent endianness. The endianness of a CPU is not visible in software only when the addressing unit of the memory is the same as the data register size of the CPU. All modern CPUs have register sizes of either 32-bit or 64-bit, while the memory is addressed in bytes, so the endianness will continue to matter in the foreseeable future.
  - moonchild 6y ago
    Registers don't have endianness. Memory does.
- deleted 6y ago
  [deleted]
- twic 6y ago
  > Most arithmetic needs to start with the less significant bits, so it's better to have them be accessible first. (E.G. most bignum representations also use little-endian, even though they don't have to.) Some operations start at the least significant end, some start at the most significant end, and it's equally easy to loop over the words in either direction, so i don't buy this. > Little endian also allows you to truncate integers more easily. What do you mean by truncate an integer? If you mean go from 32 bit to 16 bit, that's still trivial with big-endian, you just shift.
  - moonchild 6y ago
    > If you mean go from 32 bit to 16 bit, that's still trivial with big-endian, you just shift. Surely you would agree that moves are easier than shifts? Moves can be handled by the renamer and be effectively free (even for memory operands, on CISC, with the advent of memory renamers; but this is even more important for RISC, where what would otherwise have been a load followed by a shift can just be a load), whereas shifts cannot and must take up some fraction of a cycle.
- daneel_w 6y ago
  It's a legacy from days of old when it made silicon design of the memory bus easier. Arm, for example, does not incur a performance hit when switching to big-endian.
jzwinck 6y ago
Little endian has one nice feature for network communication: integer fields can be expanded or shrunk without changing code of the peer so long as the actual numbers don't overflow. Most mainstream protocols don't really take advantage of this because it is error prone, but it is used sometimes.
- kaszanka 6y ago
  I don't understand, wouldn't all of the fields after the expanded/shrunk one become misaligned?
  - jzwinck 6y ago
    It only works if you have some way to know where the next field or message starts. Sometimes you do, sometimes you don't.
gsnedders 6y ago
A lot of it is simply the prevalence of little-endian systems, initially with x86, but now with the vast majority of other systems too; code is developed on the developer's system, and works there. As big-endian systems have become increasingly rare, an ever larger amount of code has stopped working (or never worked) on big-endian systems, thus it's become harder and harder to ship big-endian systems. If you want desktop-class applications, they've typically either been little-endian only, or have been little-endian only since Apple migrated to Intel. We've seen POWER largely migrate to little-endian, as IBM was struggling with sales due to the difficulty in porting software to it, partly due to the endianness difference. (Note that POWER and PowerPC were technically always bi-endian architectures, but the majority of historic hardware had its endianness set by the motherboard in use, and almost everything shipped with big-endian. Systems running Windows NT on PowerPC were some of the very few that ran in little-endian mode.) We've also seen a similar shift in embedded hardware; bi-endian architectures like ARM and MIPS are now more commonly used in little-endian mode, despite historically being used in big-endian mode.

jzwinck 6y ago

I think the majority of binary integer data sent and received today is big endian. Calling it "legacy" seems like a stretch. People are still designing plenty of brand new protocols using big endian. When I encounter a little endian one I am pleasantly surprised, but it is still a surprise.

viraptor 6y ago
I wonder how much of that design is a conscious decision and how much is "we need to use common endianness, stuff's on the network, let's use network-endian"? If we didn't have the label of network byte order, would people default to little?
- smcameron 6y ago
  Another point is, if you use big endian on the network, and native is little endian, then if you screw up and forget to do the conversion, you catch it because it breaks. If you use little endian for native and network and you screw up, you never catch it unless you run on a big endian machine. It's a way to ensure that you implemented the protocol correctly in an endian clean way. (Of course you could also reverse the sense of the macros during testing, I suppose.)

cbmuser 6y ago

> RcouF1uZ4gsC 26 minutes ago [–] The long term good news is that the endian wars are drawing to a close. Little Endian won. I believe all the most recent CPU's support little endian at least as an option. From what I know from talking to IBM engineers, there are no plans to switch AIX on POWER or Linux on s390x to little-endian. And both are important platforms in the enterprise world.

viraptor 6y ago
Based on the number of units of those machines running in production, we can pretty safely treat them as exceptions these days. And people running them can afford to deal with changes needed for big endian conversion.
- cbmuser 6y ago
  Basically everything that is banks, governments, insurance companies used these machines. They’re not going anywhere.
  - viraptor 6y ago
    They're outnumbered by consumer level little endian devices by 6 orders of magnitude? More? Unless you know you're targeting those industries, you can likely ignore big endian. If they really want your code, you could always charge extra for big endian support too :-P I know they're not going anywhere, but they're also largely irrelevant to majority of us.

BelenusMordred 6y ago

> Little Endian won. If you could tell cryptographers this I would be very grateful.

cesarb 6y ago
They already know. Many recent cryptographic algorithms (Curve25519, ChaCha20, and so on) treat their input and output as little-endian numbers.

goalieca 6y ago

Big endian won the encoding wars with many important formats using big endian. Nearly all crypto is big endian for instance because of big number byte strings.

smcameron 6y ago

SCSI also generally uses big endian whenever there are multibyte numbers in the protocol.

Taniwha 6y ago

Strictly speaking no one uses big endian, all computers I know of are either little endian or mixed endian ('mixed. because they do strings low to high but numbers high to low). The only reason that big/mixed endian is even an issue is because some monks fucked up back when the Moors were driven from Spain ... in the wonderful libraries that were left behind they discovered Arabic numbers and realised how useful they were, far better than the clunky Roman ones - for a start a small businessman could do addition just right there on the page, writing the digits from smallest to largest as they went - sadly as I said the monks screwed up and copied the digit order from the right to left written Arabic into western languages without doing the sane thing and reversing the digits so that they continued to be written smallest to largest (so that you could do addition without knowing how much space you needed for digits until you were done). And so we're stuck with this silly artifact of history that has caused decades of arguing as we slowly rediscover the utility of putting the smaller digits first, as they were always meant to be

nabla9 6y ago
One of the main causes of the fall of the Roman Empire was that without zero they had no way to indicate successful termination of their C programs.
- Taniwha 6y ago
  that plus there was no year 0, that surely started a series of events that eventually caused the fall of the Republic
  - RcouF1uZ4gsC 6y ago
    The Republic had already fallen by that time.
fanf2 6y ago
Strings are always big endian because the most significant bit comes first when you compare them lexicographically.