7 ms·

Let's see how things have changed since 2004 when this was published: > The x86 has a small number (8) of general-purpose registers x86-64 added more general-

by comex 5y ago

Let's see how things have changed since 2004 when this was published: > The x86 has a small number (8) of general-purpose registers x86-64 added more general-purpose registers. > The x86 uses the stack to pass function parameters; the others use registers. OS vendors switched to registers for x86-64. > The x86 forgives access to unaligned data, silently fixing up the misalignment. Now ubiquitous on application processors. > The x86 has variable-sized instructions. The others use fixed-sized instructions. ARM introduced Thumb-2, with a mix of 2-byte and 4-byte instructions, in 2003. PowerPC and RISC-V also added some form of variable-length instruction support. On the other hand, ARM turned around and dropped variable-length instructions with its 64-bit architecture released in 2011. > The x86 has a strict memory model … The others have weak memory models Still x86-only. > The x86 supports atomic load-modify-store operations. None of the others do. As opposed to load-linked/store-conditional, which is a different way to express the same basic idea? Or is he claiming that other processors didn't support any form of atomic instructions, which definitely isn't true? At any rate, ARM previously had load-linked/store-conditional but recently added a native compare-and-swap instruction with ARMv8.1. > The x86 passes function return addresses on the stack. The others use a link register. Still x86-only.

ungamedplayer 5y ago
Thank you for writing this. I was going to cover quite a lot of these points and you have done it so very succinctly. It may be obvious, but I think it bears repeating. This blog entry should not reflect badly on Raymond C as he was reporting on the architecture at this time.
- zinekeller 5y ago
  The 2022 follow-up also said "And by x86 I mean specifically x86-32." Also, I don't he was on the AMD64 team yet at that time (still Itanium) so probably that is something.
ncmncm 5y ago
Apple M1 supports optional x86-style memory event ordering, so that its x86 emulation could be made to work without penalty. When SPARC got new microcode supporting unaligned access, it turned out to be a big performance win, as the alignment padding had made for a bigger cache footprint. That was an embarrassment for the whole RISC industry. Nobody today would field a chip that enforced alignment. The alignment penalty might have been smaller back when clock rates were closer to memory latency, but caches were radically smaller then, too, so even more affected by inflated footprint.
- jnordwick 5y ago
  > as the alignment padding had made for a bigger cache footprint I argues with some of the Rust compiler members the other day about wanting to just ditch almost all alighnment restrictions because I of this exact thing. They laughed and basically told me i didnt know what i was talking about. I remember about 15 years ago when i worked at market making firm and we test this and it was a great gain - we started packing almost all our structs after that. Now, at another MM shop, and we're trying to push the same thing but having to fight these areguments again (the only alignmets I want to keep are for AVX and hardware accessed buffers).
  - ncmncm 5y ago
    Superstition is as powerful as it ever was.
    - MBCook 5y ago
      Is this superstition or more received wisdom, which may have been true at one point in the past as is now just orthodoxy?
      - notriddle 5y ago
        Fifty bucks says it isn't even about performance, but is instead about passing pointers to C code. Zero-overhead FFI has killed a lot of radical performance improvements that Rust could have otherwise made. I don't know, because nobody's actually posting a link to it.
NtGuy25 5y ago
In regards to memory alignment, it's even worse. Most instructions work on unaligned data. But some instructions require 8 byte, 16 byte, 32 byte, 64 byte and I think there's even some 128 and 256 byte alignment. One of the more common pitfalls someone can find themselves in when coding x86-64 asm.
ceeplusplus 5y ago
There is still one big thing that hasn't changed but has been the subject of discussion on whether x86-64 is fundamentally bottlenecking CPU architecture. Variable length instructions means decoder complexity scales quadratically rather than linearly. It's been speculated this is one reason why even the latest x86 architectures stick with relatively narrow decode but Arm CPUs with lower performance levels (e.g. Cortex X1/2) are already 5-wide and Apple is 8-wide.
cesarb 5y ago
> ARM introduced Thumb-2, with a mix of 2-byte and 4-byte instructions, in 2003. PowerPC and RISC-V also [...] x86 is still the weirdo. Both Thumb-2 and the RISC-V C extension (I don't know about PowerPC) have only 2-byte and 4-byte instructions, aligned to 2 bytes; x86 instructions can vary from 1 to 15 bytes, with no alignment requirement.
- classichasclass 5y ago
  Power10 has prefixed instructions. These are essentially 64-bit instructions in two pieces. They are odd even (particularly?) to those of us who have worked with the architecture for a long time, and not much otherwise supports them yet. Their motivation is primarily to more efficiently represent constants and offsets.
- cryptonector 5y ago
  I suspect variable-length instructions are a big gain because you get to pack instructions more tightly and so have fewer cache misses. Though, obviously, it's going to depend on having an instruction set that yields shorter text for typical assembly than fixed-sized instructions would. (In a way, opcodes need a bit of Huffman encoding!) Any losses from having to do more decoding work are probably offset by having sufficiently deep pipelines and enough decoders.
  - phdelightful 5y ago
    The counterpoint is that variable-length decoding introduces sequential dependence in the decoding, i.e. you don’t know where instruction 2 starts until you’ve decoded instruction 1. This probably limits how many decoders you can have. If you know all your instructions are 4B you can basically decode as many as you want in parallel.
    - gpderetta 5y ago
      I think what happen in practice is that the decoders still speculatively decode in parallel then drop all misdecoded instructions. Easy when instructions have only a couple of sizes. Hard and wasteful for something like x86.
phamilton 5y ago
I vaguely recall that LL/SC solves the ABA problem whereas load-modify-store does not. It's been a while, so I'm going to define my understanding of the ABA problem in case I misunderstood it: x86 only supplies cmpxchange instructions on which will update a value only if it matches the passed in previous value. There's a class of concurrency bugs where the value is modified away from it's initial value and then modified again back to it's value. cmpxchange can't detect that condition, so if it's a meaningful difference often the 128-bit cmpxchange will be used with a counter in the second 64-bits that is incremented on each write to catch this case. LL/SC will trigger on any write, rather than comparing the value, providing the stronger guarantee. (Please correct me if this is inaccurate, it's been a hit minute since I learned this and I'd love to be more current on it).
- zozbot234 5y ago
  AIUI, a cmpxchg loop is enough to implement read-modify-write of any atomically sized value. The ABA problem becomes relevant when trying to implement more complex lock-free data structures.
wolpoli 5y ago
Thanks for summarizing this. Did they do any other clean-up when moving to 64 bit?
- moonchild 5y ago
  PC-relative code.
gpderetta 5y ago
>>The x86 has a strict memory model … The others have weak memory models > Still x86-only. SPARC supported (eventually exclusively) TSO well before x86 committed to it. For a while Intel claimed to support some form of Processor Ordering which I understand is slightly weaker than TSO, although no Intel CPU ever took advantage of the weakened constraints.
moonchild 5y ago
>> The x86 supports atomic load-modify-store operations. None of the others do. > As opposed to load-linked/store-conditional, which is a different way to express the same basic idea? Perhaps that x86 has atomic addition, where other architectures must use a cas/ll+sc loop?
ttybird2 5y ago
"As opposed to load-linked/store-conditional, which is a different way to express the same basic idea? Or is he claiming that other processors didn't support any form of atomic instructions, which definitely isn't true?" It refers specifically to things like fetch_and_add. Supported by risc-v and i64.

by comex 5y ago

ungamedplayer 5y ago

Thank you for writing this. I was going to cover quite a lot of these points and you have done it so very succinctly. It may be obvious, but I think it bears repeating. This blog entry should not reflect badly on Raymond C as he was reporting on the architecture at this time.

zinekeller 5y ago
The 2022 follow-up also said "And by x86 I mean specifically x86-32." Also, I don't he was on the AMD64 team yet at that time (still Itanium) so probably that is something.

ncmncm 5y ago

Apple M1 supports optional x86-style memory event ordering, so that its x86 emulation could be made to work without penalty. When SPARC got new microcode supporting unaligned access, it turned out to be a big performance win, as the alignment padding had made for a bigger cache footprint. That was an embarrassment for the whole RISC industry. Nobody today would field a chip that enforced alignment. The alignment penalty might have been smaller back when clock rates were closer to memory latency, but caches were radically smaller then, too, so even more affected by inflated footprint.

jnordwick 5y ago
> as the alignment padding had made for a bigger cache footprint I argues with some of the Rust compiler members the other day about wanting to just ditch almost all alighnment restrictions because I of this exact thing. They laughed and basically told me i didnt know what i was talking about. I remember about 15 years ago when i worked at market making firm and we test this and it was a great gain - we started packing almost all our structs after that. Now, at another MM shop, and we're trying to push the same thing but having to fight these areguments again (the only alignmets I want to keep are for AVX and hardware accessed buffers).
- ncmncm 5y ago
  Superstition is as powerful as it ever was.
  - MBCook 5y ago
    Is this superstition or more received wisdom, which may have been true at one point in the past as is now just orthodoxy?
    - notriddle 5y ago
      Fifty bucks says it isn't even about performance, but is instead about passing pointers to C code. Zero-overhead FFI has killed a lot of radical performance improvements that Rust could have otherwise made. I don't know, because nobody's actually posting a link to it.

NtGuy25 5y ago

In regards to memory alignment, it's even worse. Most instructions work on unaligned data. But some instructions require 8 byte, 16 byte, 32 byte, 64 byte and I think there's even some 128 and 256 byte alignment. One of the more common pitfalls someone can find themselves in when coding x86-64 asm.

ceeplusplus 5y ago

There is still one big thing that hasn't changed but has been the subject of discussion on whether x86-64 is fundamentally bottlenecking CPU architecture. Variable length instructions means decoder complexity scales quadratically rather than linearly. It's been speculated this is one reason why even the latest x86 architectures stick with relatively narrow decode but Arm CPUs with lower performance levels (e.g. Cortex X1/2) are already 5-wide and Apple is 8-wide.

cesarb 5y ago

> ARM introduced Thumb-2, with a mix of 2-byte and 4-byte instructions, in 2003. PowerPC and RISC-V also [...] x86 is still the weirdo. Both Thumb-2 and the RISC-V C extension (I don't know about PowerPC) have only 2-byte and 4-byte instructions, aligned to 2 bytes; x86 instructions can vary from 1 to 15 bytes, with no alignment requirement.

classichasclass 5y ago
Power10 has prefixed instructions. These are essentially 64-bit instructions in two pieces. They are odd even (particularly?) to those of us who have worked with the architecture for a long time, and not much otherwise supports them yet. Their motivation is primarily to more efficiently represent constants and offsets.
cryptonector 5y ago
I suspect variable-length instructions are a big gain because you get to pack instructions more tightly and so have fewer cache misses. Though, obviously, it's going to depend on having an instruction set that yields shorter text for typical assembly than fixed-sized instructions would. (In a way, opcodes need a bit of Huffman encoding!) Any losses from having to do more decoding work are probably offset by having sufficiently deep pipelines and enough decoders.
- phdelightful 5y ago
  The counterpoint is that variable-length decoding introduces sequential dependence in the decoding, i.e. you don’t know where instruction 2 starts until you’ve decoded instruction 1. This probably limits how many decoders you can have. If you know all your instructions are 4B you can basically decode as many as you want in parallel.
  - gpderetta 5y ago
    I think what happen in practice is that the decoders still speculatively decode in parallel then drop all misdecoded instructions. Easy when instructions have only a couple of sizes. Hard and wasteful for something like x86.

phamilton 5y ago

I vaguely recall that LL/SC solves the ABA problem whereas load-modify-store does not. It's been a while, so I'm going to define my understanding of the ABA problem in case I misunderstood it: x86 only supplies cmpxchange instructions on which will update a value only if it matches the passed in previous value. There's a class of concurrency bugs where the value is modified away from it's initial value and then modified again back to it's value. cmpxchange can't detect that condition, so if it's a meaningful difference often the 128-bit cmpxchange will be used with a counter in the second 64-bits that is incremented on each write to catch this case. LL/SC will trigger on any write, rather than comparing the value, providing the stronger guarantee. (Please correct me if this is inaccurate, it's been a hit minute since I learned this and I'd love to be more current on it).

zozbot234 5y ago
AIUI, a cmpxchg loop is enough to implement read-modify-write of any atomically sized value. The ABA problem becomes relevant when trying to implement more complex lock-free data structures.

wolpoli 5y ago

Thanks for summarizing this. Did they do any other clean-up when moving to 64 bit?

moonchild 5y ago
PC-relative code.

gpderetta 5y ago

>>The x86 has a strict memory model … The others have weak memory models > Still x86-only. SPARC supported (eventually exclusively) TSO well before x86 committed to it. For a while Intel claimed to support some form of Processor Ordering which I understand is slightly weaker than TSO, although no Intel CPU ever took advantage of the weakened constraints.

moonchild 5y ago

>> The x86 supports atomic load-modify-store operations. None of the others do. > As opposed to load-linked/store-conditional, which is a different way to express the same basic idea? Perhaps that x86 has atomic addition, where other architectures must use a cas/ll+sc loop?

ttybird2 5y ago

"As opposed to load-linked/store-conditional, which is a different way to express the same basic idea? Or is he claiming that other processors didn't support any form of atomic instructions, which definitely isn't true?" It refers specifically to things like fetch_and_add. Supported by risc-v and i64.