5 ms·

It was none faster because it was impossible to program, without the ability to load or store anything but natively aligned and sized quads and with a quite use

It was none faster because it was impossible to program, without the ability to load or store anything but natively aligned and sized quads and with a quite useless memory order model (essentially anything could be reordered past any other thing). Reading, modifying, and writing one byte on this rig was basically impossible and don’t even think about how hard it would be to write a mutex.

dragontamer 7y ago
> essentially anything could be reordered past any other thing Except for memory barriers. The DEC Alpha command "mb". x86 has a (relatively) strong memory model, but ARM and POWER9 both have weak memory models. Not quite as weak as DEC Alpha, but weak enough that you need to be very careful about memory-barrier placement in both ARM and POWER9. XBox360, PS3, and ARM (cell phone) system programmers would know about the difficulties involved. Yeah, its hard to learn, but totally possible to write a mutex. -------------- There's probably more code running on "weak memory models" today (ie ARM) than on "strong memory models" (aka: x86) today. > without the ability to load or store anything but natively aligned and sized quads Check out the assembly code generated with -O3 with GCC or LLVM. x86 has natively aligned quads for performance reasons, and compilers have also solved this problem. In fact, you'll see plenty of "nop" generated in compiler code, so that all of your assembly is aligned to the cache-line to maximize uop cache issues on modern x86 machines. x86 doesn't have any REQUIREMENT for aligned assembly code or data. But x86 is far more efficient when reading/writing from cache-aligned locations. (In particular: reading across an 64-byte boundary naturally results in 2x L1 cache reads instead of 1x L1 cache read). Both the alignment, and memory-barrier, issue are solved today. Arguably, they weren't solved back in the DEC Alpha days (I was too young to be programming at that time)... but modern compilers and toolchains can certainly work with memory barriers and "aligned only" memory.
- BeeOnRope 7y ago
  On the second point, the OP is complaining more a about the lack of a way to directly access anything smaller than 4 bytes, i.e., that there are no 1 or 2 byte reads or writes. In addition to being slow for e.g. text processing, you couldn't implement a modern memory model like Java's or C++ on it, without serious compromises, due to the inability to make non-interfering writes of adjacent elements.
  - dragontamer 7y ago
    > In addition to being slow for e.g. text processing, you couldn't implement a modern memory model like Java's or C++ on it, without serious compromises, due to the inability to make non-interfering writes of adjacent elements. But consider that the modern model accounts for 64-byte (aka: 512-bit) cachelines. Any reads or writes to a 64-byte cacheline is going to be false-shared between cores. From a performance point of view, you cannot make non-interfering writes of adjacent elements in the modern memory model! If you're dealing with single-threaded code, then bit-mask instructions for registers should be sufficient to emulate the ability. I can see that bit-masking every character instruction would be slower, but it would certainly get the job done.
    - BeeOnRope 7y ago
      I don't think the performance thing is relevant here. First you have to be able to implement the model correctly. That means non-interfering writes. 99.99% of those writes will never involve any sharing (false or otherwise) and will perform just fine. Not being able to do byte-granular writes means you simply can't implement things like this at all unless you: 1) Decide that "char" will be 32 bits (i.e., make a byte on your system 4x larger than usual). 2) Do some locking around each narrow write (or perhaps a series of them) and ensure everyone does the same. So I am not really following your argument with regard to false sharing. I don't know what you mean by "the modern model accounts for 64-byte (aka: 512-bit) cachelines". I don't know of any (software) memory model that even mentions cache lines. Hardware memory models don't lean heavily on them either, although they do get mentioned (e.g., on x86, Intel provides additional guarantees for some misaligned reads that do not cross a cache line, and AMD provides different guarantees).
      - dragontamer 7y ago
        > 1) Decide that "char" will be 32 bits (i.e., make a byte on your system 4x larger than usual). 2) Do some locking around each narrow write (or perhaps a series of them) and ensure everyone does the same. Or 3) Decide that for performance reasons, reads/writes to an array (even a char* array or string) would be locked in 64-byte chunks at a minimum. My point with false-sharing is that "true" byte-granularity comes with severe performance penalties on modern x86 systems. Its a thing that you want to avoid. If one thread is reading / writing to char myString[0], while a 2nd thread is reading/writing to myString[63], your memory will ping-pong between the two threads (because x86 shares 64-bytes at a time between caches). -------- I guess your point is that the CPU will "do the right thing, even if they're forced to do it slowly" in this case. So I guess you're right about that. Your code CAN be written as multithreaded + byte granular, its just going to be slow in practice. I guess my point is that the high-performance programmer needs to understand this situation anyway, and that high-performance programmers have developed methodologies to avoid false-sharing. Any high-performance data-structure needs to understand false-sharing. You really shouldn't be doing things like reading/writing to myString[0] and myString[63] in two separate threads.
- pizlonator 7y ago
  ARM’s memory model is stronger than Alpha’s especially on ARM64. It’s not about just whether it’s possible to write a mutex but also whether you can write a really good one. The best concurrent algorithms - whether mutexes or lock-free data structures - benefit from a careful mix of strength and weakness in the memory model. I think Alpha is too weak. X86 may be too strong - so say smart people - but it still manages to be fast as fuck. No idea what you’re talking about wrt alignment. On x86, you can load/store misaligned. On Alpha, you can’t. On ARM, you can on some but not on others. The CPUs where you can are easier to program. No idea what you’re talking about wrt memory barriers. It’s not a solved problem. Memory barriers slow things down so it’s better not to have to use them. X86 and ARM64 give you tricks to avoid using barriers, or to use cheap barriers, in many important racy algorithms. Alpha gives you fewer tricks. (Specifically x86 gives you lots of ordering “for free” while arm64 let’s you use the self-xor dependency trick to cheaply order loads and has a generous buffet of half fences.) The memory model matters a lot - the weaker it is, the fewer tricks programmers have to avoid doing expensive things to request specific orderings.
  - dragontamer 7y ago
    > On x86, you can load/store misaligned. At a performance penalty. Read or write across a 64-byte cacheline, and your CPU will be forced to perform 2x loads to implement your singular unaligned load. An unaligned load across a cacheline is literally 1/2 the speed of an aligned load, and is something the modern programmer (and compiler) is designed to avoid. The compiler basically avoids all misaligned loads/stores, even on x86. (Aside from when the programmer really forces it: like reinterpret_cast<int>(0x800003f) or something) ------- > The best concurrent algorithms - whether mutexes or lock-free data structures - benefit from a careful mix of strength and weakness in the memory model. And all of those algorithms would run CORRECTLY, if those half-barriers were replaced with full barriers. Maybe slower, but they'd be correct. That "generous buffet of half fences" can only exist on a weak memory-model system (like ARM), because x86 AUTOMATICALLY performs those half-fences after every load / before every store instruction. That's the thing about strong memory models: once you're "too strong", it doesn't even make sense to have those half-barrier instructions. ARM and PowerPC were too weak 10-years ago. They've "strengthened" their memory model by adding new half-barriers to their instruction set. That's the real secret: to change your CPU over time to match programmer's preferences. ARM / PowerPC started off too weak, but are now approaching "just right", with the addition of new instructions. DEC Alpha can't do that, because the DEC company died decades ago. Its only fair to consider the programming environment and expectations of the time period that the DEC Alpha existed in.
- BeeOnRope 7y ago
  > Check out the assembly code generated with -O3 with GCC or LLVM. x86 has natively aligned quads for performance reasons, and compilers have also solved this problem. You make it sound like a compiler choice (that possibly only occurs when certain optimization levels are used). In fact, it is part of the language spec: everything has to be properly aligned based on the C and C++ rules. This leaks into the ABI too when it comes to structure packing, etc. So alignment is pervasive because those are the rules, and the rules were made for performance, e.g., to support platforms like Alpha which couldn't even _do_ an unaligned read but would have to construct the value from two aligned reads. > x86 doesn't have any REQUIREMENT for aligned assembly code or data. But x86 is far more efficient when reading/writing from cache-aligned locations. (In particular: reading across an 64-byte boundary naturally results in 2x L1 cache reads instead of 1x L1 cache read). x86 is not far more efficient when reading/writing aligned locations, in most cases! One of the performance secrets is that unaligned access is really fast in modern x86. There is no penalty at all if the load doesn't cross a cache line, and especially for the 4 and 8 byte reads we are talking about, that would be the usual case. Even when they cross, they are just half as fast: so "only" 1 load/cycle, rather than 2. Yes, 1 unaligned load every cycle. A lot of code is doing less than 1 load/cycle anyways, so you may not bump up against this limit. So average those types of reads might be a few % slower in a theoretical sense (due to the occasional line crossing), and often close to zero in reality. Of course you could design pathological cases, like page crossing every load... > x86 doesn't have any REQUIREMENT for aligned assembly code or data For data, it does, e.g., any SSE instruction with a memory operand, or any of the "aligned" load variants like vmovdqa. People who say "x86 doesn't need alignment" and create unaligned data will eventually get bitten when a compiler inserts one of those instructions for you. So even x86 compilation relies on the language-level alignment rules.

dragontamer 7y ago

> essentially anything could be reordered past any other thing Except for memory barriers. The DEC Alpha command "mb". x86 has a (relatively) strong memory model, but ARM and POWER9 both have weak memory models. Not quite as weak as DEC Alpha, but weak enough that you need to be very careful about memory-barrier placement in both ARM and POWER9. XBox360, PS3, and ARM (cell phone) system programmers would know about the difficulties involved. Yeah, its hard to learn, but totally possible to write a mutex. -------------- There's probably more code running on "weak memory models" today (ie ARM) than on "strong memory models" (aka: x86) today. > without the ability to load or store anything but natively aligned and sized quads Check out the assembly code generated with -O3 with GCC or LLVM. x86 has natively aligned quads for performance reasons, and compilers have also solved this problem. In fact, you'll see plenty of "nop" generated in compiler code, so that all of your assembly is aligned to the cache-line to maximize uop cache issues on modern x86 machines. x86 doesn't have any REQUIREMENT for aligned assembly code or data. But x86 is far more efficient when reading/writing from cache-aligned locations. (In particular: reading across an 64-byte boundary naturally results in 2x L1 cache reads instead of 1x L1 cache read). Both the alignment, and memory-barrier, issue are solved today. Arguably, they weren't solved back in the DEC Alpha days (I was too young to be programming at that time)... but modern compilers and toolchains can certainly work with memory barriers and "aligned only" memory.

BeeOnRope 7y ago
On the second point, the OP is complaining more a about the lack of a way to directly access anything smaller than 4 bytes, i.e., that there are no 1 or 2 byte reads or writes. In addition to being slow for e.g. text processing, you couldn't implement a modern memory model like Java's or C++ on it, without serious compromises, due to the inability to make non-interfering writes of adjacent elements.
- dragontamer 7y ago
  > In addition to being slow for e.g. text processing, you couldn't implement a modern memory model like Java's or C++ on it, without serious compromises, due to the inability to make non-interfering writes of adjacent elements. But consider that the modern model accounts for 64-byte (aka: 512-bit) cachelines. Any reads or writes to a 64-byte cacheline is going to be false-shared between cores. From a performance point of view, you cannot make non-interfering writes of adjacent elements in the modern memory model! If you're dealing with single-threaded code, then bit-mask instructions for registers should be sufficient to emulate the ability. I can see that bit-masking every character instruction would be slower, but it would certainly get the job done.
  - BeeOnRope 7y ago
    I don't think the performance thing is relevant here. First you have to be able to implement the model correctly. That means non-interfering writes. 99.99% of those writes will never involve any sharing (false or otherwise) and will perform just fine. Not being able to do byte-granular writes means you simply can't implement things like this at all unless you: 1) Decide that "char" will be 32 bits (i.e., make a byte on your system 4x larger than usual). 2) Do some locking around each narrow write (or perhaps a series of them) and ensure everyone does the same. So I am not really following your argument with regard to false sharing. I don't know what you mean by "the modern model accounts for 64-byte (aka: 512-bit) cachelines". I don't know of any (software) memory model that even mentions cache lines. Hardware memory models don't lean heavily on them either, although they do get mentioned (e.g., on x86, Intel provides additional guarantees for some misaligned reads that do not cross a cache line, and AMD provides different guarantees).
    - dragontamer 7y ago
      > 1) Decide that "char" will be 32 bits (i.e., make a byte on your system 4x larger than usual). 2) Do some locking around each narrow write (or perhaps a series of them) and ensure everyone does the same. Or 3) Decide that for performance reasons, reads/writes to an array (even a char* array or string) would be locked in 64-byte chunks at a minimum. My point with false-sharing is that "true" byte-granularity comes with severe performance penalties on modern x86 systems. Its a thing that you want to avoid. If one thread is reading / writing to char myString[0], while a 2nd thread is reading/writing to myString[63], your memory will ping-pong between the two threads (because x86 shares 64-bytes at a time between caches). -------- I guess your point is that the CPU will "do the right thing, even if they're forced to do it slowly" in this case. So I guess you're right about that. Your code CAN be written as multithreaded + byte granular, its just going to be slow in practice. I guess my point is that the high-performance programmer needs to understand this situation anyway, and that high-performance programmers have developed methodologies to avoid false-sharing. Any high-performance data-structure needs to understand false-sharing. You really shouldn't be doing things like reading/writing to myString[0] and myString[63] in two separate threads.
pizlonator 7y ago
ARM’s memory model is stronger than Alpha’s especially on ARM64. It’s not about just whether it’s possible to write a mutex but also whether you can write a really good one. The best concurrent algorithms - whether mutexes or lock-free data structures - benefit from a careful mix of strength and weakness in the memory model. I think Alpha is too weak. X86 may be too strong - so say smart people - but it still manages to be fast as fuck. No idea what you’re talking about wrt alignment. On x86, you can load/store misaligned. On Alpha, you can’t. On ARM, you can on some but not on others. The CPUs where you can are easier to program. No idea what you’re talking about wrt memory barriers. It’s not a solved problem. Memory barriers slow things down so it’s better not to have to use them. X86 and ARM64 give you tricks to avoid using barriers, or to use cheap barriers, in many important racy algorithms. Alpha gives you fewer tricks. (Specifically x86 gives you lots of ordering “for free” while arm64 let’s you use the self-xor dependency trick to cheaply order loads and has a generous buffet of half fences.) The memory model matters a lot - the weaker it is, the fewer tricks programmers have to avoid doing expensive things to request specific orderings.
- dragontamer 7y ago
  > On x86, you can load/store misaligned. At a performance penalty. Read or write across a 64-byte cacheline, and your CPU will be forced to perform 2x loads to implement your singular unaligned load. An unaligned load across a cacheline is literally 1/2 the speed of an aligned load, and is something the modern programmer (and compiler) is designed to avoid. The compiler basically avoids all misaligned loads/stores, even on x86. (Aside from when the programmer really forces it: like reinterpret_cast<int>(0x800003f) or something) ------- > The best concurrent algorithms - whether mutexes or lock-free data structures - benefit from a careful mix of strength and weakness in the memory model. And all of those algorithms would run CORRECTLY, if those half-barriers were replaced with full barriers. Maybe slower, but they'd be correct. That "generous buffet of half fences" can only exist on a weak memory-model system (like ARM), because x86 AUTOMATICALLY performs those half-fences after every load / before every store instruction. That's the thing about strong memory models: once you're "too strong", it doesn't even make sense to have those half-barrier instructions. ARM and PowerPC were too weak 10-years ago. They've "strengthened" their memory model by adding new half-barriers to their instruction set. That's the real secret: to change your CPU over time to match programmer's preferences. ARM / PowerPC started off too weak, but are now approaching "just right", with the addition of new instructions. DEC Alpha can't do that, because the DEC company died decades ago. Its only fair to consider the programming environment and expectations of the time period that the DEC Alpha existed in.
BeeOnRope 7y ago
> Check out the assembly code generated with -O3 with GCC or LLVM. x86 has natively aligned quads for performance reasons, and compilers have also solved this problem. You make it sound like a compiler choice (that possibly only occurs when certain optimization levels are used). In fact, it is part of the language spec: everything has to be properly aligned based on the C and C++ rules. This leaks into the ABI too when it comes to structure packing, etc. So alignment is pervasive because those are the rules, and the rules were made for performance, e.g., to support platforms like Alpha which couldn't even _do_ an unaligned read but would have to construct the value from two aligned reads. > x86 doesn't have any REQUIREMENT for aligned assembly code or data. But x86 is far more efficient when reading/writing from cache-aligned locations. (In particular: reading across an 64-byte boundary naturally results in 2x L1 cache reads instead of 1x L1 cache read). x86 is not far more efficient when reading/writing aligned locations, in most cases! One of the performance secrets is that unaligned access is really fast in modern x86. There is no penalty at all if the load doesn't cross a cache line, and especially for the 4 and 8 byte reads we are talking about, that would be the usual case. Even when they cross, they are just half as fast: so "only" 1 load/cycle, rather than 2. Yes, 1 unaligned load every cycle. A lot of code is doing less than 1 load/cycle anyways, so you may not bump up against this limit. So average those types of reads might be a few % slower in a theoretical sense (due to the occasional line crossing), and often close to zero in reality. Of course you could design pathological cases, like page crossing every load... > x86 doesn't have any REQUIREMENT for aligned assembly code or data For data, it does, e.g., any SSE instruction with a memory operand, or any of the "aligned" load variants like vmovdqa. People who say "x86 doesn't need alignment" and create unaligned data will eventually get bitten when a compiler inserts one of those instructions for you. So even x86 compilation relies on the language-level alignment rules.