8 ms·

Inlining heuristics used for -O3 are architecture-specific, so this just shows that things have been tuned in a particular way, probably based on the most preva

Inlining heuristics used for -O3 are architecture-specific, so this just shows that things have been tuned in a particular way, probably based on the most prevalent icache sizes. -O2 numbers might be different because of reduced inlining. I would expect the -O2 numbers to reflect better the actual ISA capabilities. The choice of sub-architecture matters as well. It's probably not fair to compare against outline atomics. In my experiments at least, VEX was more compact even for scalar code, etc.

inkyoto 4y ago
> Inlining heuristics used for -O3 are architecture-specific […] Yes, the inlining strategy is non-deterministic and does not yield same results across different ISA. Moreover, it does not even guarantee the same result for a single ISA if the code was compiled for a specific submodel of a CPU with a different instruction cache line size. Especially for the x86 ISA where instructions are variable length, it is common for instructions to inadvertantly spill over into the next I-cache line thus yielding a substantial performance penalty for a performance critical/sensitive code path. Therefore, a common technique for the optimiser was (I have not checked recently tho) to take into account the I-cache line size, group instructions in such a way that if a fat code sequence were to cross the cache line, fill in the rest of the cache line with NOP's and place the fat instruction into the next cache line. Such a problem is nearly non-existant for RISC ISA's although I would surmise that one has to watch out for tight loops anyway. > I would expect the -O2 numbers to reflect better the actual ISA capabilities. I would go on to add that today «-O3 -fno-inline-functions» would give a more accurate and faithful reflection of generic ISA capabilities today. For a long time, «-O3» was no more than «-O2 -finline-functions»; however, since then further optimisations have been added to «-O3» that are rather useful generic optimisations for modern CPU's (i.e. loop vectorisation and more). The article is especially lacklustre in this particular regard as the author does not go beyond generic bloviations and does not make an attempt to understand what hides beyond «-O3».
mbitsnbites 4y ago
The article has been updated with -O2 numbers. Spoiler alert: ARM & RISC-V still win over x86 & z/Arch.
ajross 4y ago
Came here to say exactly this. And in fact x86_64 has historically gotten a ton more tuning and (at least in my experience) will more aggressively inline at pretty much all optimization levels. I really don't know why an article about code density chose that level instead of -Os or -O2.
- mbitsnbites 4y ago
  Simple: Optimizing for small code size is uninteresting for 99% of the software that a CPU runs. What matters is performance (especially when you argue that dense code is good for performance), and most real world software is optimized for performance. I get the argument of -O2 vs -O3 and I'll be sure to measure that. However my experience is that x86_64 does pretty poorly in -O2 too compared to RISC.
  - gpderetta 4y ago
    Sure, but a static code size analysis has very very little to do with performance as you will be weighting equally code that is executed millions of times with code that is never executed.
    - mbitsnbites 4y ago
      I agree, but in absence of an accurate method this felt like the second best method. Then there's also the elephant in the room that I'd like to write an article about some time: The decoder/translator + uop-cache in the front end has a devastating effect on instruction fetch & decode performance. That silicon eats power, could be used for better things (larger L1I cache etc), adds latency, limits how wide you can decode, and so on. Rationale: CISC is not just about density. With good RISC you can get much better fetch & decode bandwidth (all other things being equal). E.g. see Apple silicon.
      - gpderetta 4y ago
        I don't think that x86 implementations are transistor limited. In fact Intel had to slap giant vector ALUs to find an use for them. And x86 L1 size is unfortunately limited by the page size, so you can't really trade one for the other. Complex decoders do consume power of course, but I don't think they have a huge effect on the thermal budget. I also don't think they have a huge effect on latency and the uop L0 cache actually improves latency. They make it harder to scale to higher width of course, but it seems that it hasn't been a huge obstacle so far.
mbitsnbites 4y ago
> I would expect the -O2 numbers to reflect better the actual ISA capabilities. I will have a more thorough look at -O2 too, good point. However w.r.t. "ISA capabilities" the simple fact is that compilers don't use CISC style instructions. They prefer RISC style load/store + many GPRs code, and new x86_64 instructions are modelled after this paradigm. Hence I wouldn't expect there to be a huge difference at -O2 (and in my experience x86_64 often loses at -O2 too).
- fweimer 4y ago
  I think for x86-64, the 32-bit immediate operands totally qualify as CISC-style because they contribute to the variable length nature of the instruction encoding. And GCC uses them all the time, of course. GCC has a surprising tendency to use the string instructions for memset and memcpy, too. In the other direction, without -march=x86-64-v3, GCC cannot even use most of the more RISC-style three-operand instructions.
  - mbitsnbites 4y ago
    The 32-bit operands are actually hurting code density most of the time. An instruction that would take 4 bytes in a RISC ISA takes at least 5 bytes in a CISC ISA. Same thing with the 32-bit offsets for x86_64 conditional branches. Edit: With "CISC style" I meant those instructions that use implicit registers instead of GPRs and/or do fancy multi-operation stuff, like LOOP or POPA/PUSHA etc.

inkyoto 4y ago

> Inlining heuristics used for -O3 are architecture-specific […] Yes, the inlining strategy is non-deterministic and does not yield same results across different ISA. Moreover, it does not even guarantee the same result for a single ISA if the code was compiled for a specific submodel of a CPU with a different instruction cache line size. Especially for the x86 ISA where instructions are variable length, it is common for instructions to inadvertantly spill over into the next I-cache line thus yielding a substantial performance penalty for a performance critical/sensitive code path. Therefore, a common technique for the optimiser was (I have not checked recently tho) to take into account the I-cache line size, group instructions in such a way that if a fat code sequence were to cross the cache line, fill in the rest of the cache line with NOP's and place the fat instruction into the next cache line. Such a problem is nearly non-existant for RISC ISA's although I would surmise that one has to watch out for tight loops anyway. > I would expect the -O2 numbers to reflect better the actual ISA capabilities. I would go on to add that today «-O3 -fno-inline-functions» would give a more accurate and faithful reflection of generic ISA capabilities today. For a long time, «-O3» was no more than «-O2 -finline-functions»; however, since then further optimisations have been added to «-O3» that are rather useful generic optimisations for modern CPU's (i.e. loop vectorisation and more). The article is especially lacklustre in this particular regard as the author does not go beyond generic bloviations and does not make an attempt to understand what hides beyond «-O3».

mbitsnbites 4y ago

The article has been updated with -O2 numbers. Spoiler alert: ARM & RISC-V still win over x86 & z/Arch.

ajross 4y ago

Came here to say exactly this. And in fact x86_64 has historically gotten a ton more tuning and (at least in my experience) will more aggressively inline at pretty much all optimization levels. I really don't know why an article about code density chose that level instead of -Os or -O2.

mbitsnbites 4y ago
Simple: Optimizing for small code size is uninteresting for 99% of the software that a CPU runs. What matters is performance (especially when you argue that dense code is good for performance), and most real world software is optimized for performance. I get the argument of -O2 vs -O3 and I'll be sure to measure that. However my experience is that x86_64 does pretty poorly in -O2 too compared to RISC.
- gpderetta 4y ago
  Sure, but a static code size analysis has very very little to do with performance as you will be weighting equally code that is executed millions of times with code that is never executed.
  - mbitsnbites 4y ago
    I agree, but in absence of an accurate method this felt like the second best method. Then there's also the elephant in the room that I'd like to write an article about some time: The decoder/translator + uop-cache in the front end has a devastating effect on instruction fetch & decode performance. That silicon eats power, could be used for better things (larger L1I cache etc), adds latency, limits how wide you can decode, and so on. Rationale: CISC is not just about density. With good RISC you can get much better fetch & decode bandwidth (all other things being equal). E.g. see Apple silicon.
    - gpderetta 4y ago
      I don't think that x86 implementations are transistor limited. In fact Intel had to slap giant vector ALUs to find an use for them. And x86 L1 size is unfortunately limited by the page size, so you can't really trade one for the other. Complex decoders do consume power of course, but I don't think they have a huge effect on the thermal budget. I also don't think they have a huge effect on latency and the uop L0 cache actually improves latency. They make it harder to scale to higher width of course, but it seems that it hasn't been a huge obstacle so far.

mbitsnbites 4y ago

> I would expect the -O2 numbers to reflect better the actual ISA capabilities. I will have a more thorough look at -O2 too, good point. However w.r.t. "ISA capabilities" the simple fact is that compilers don't use CISC style instructions. They prefer RISC style load/store + many GPRs code, and new x86_64 instructions are modelled after this paradigm. Hence I wouldn't expect there to be a huge difference at -O2 (and in my experience x86_64 often loses at -O2 too).

fweimer 4y ago
I think for x86-64, the 32-bit immediate operands totally qualify as CISC-style because they contribute to the variable length nature of the instruction encoding. And GCC uses them all the time, of course. GCC has a surprising tendency to use the string instructions for memset and memcpy, too. In the other direction, without -march=x86-64-v3, GCC cannot even use most of the more RISC-style three-operand instructions.
- mbitsnbites 4y ago
  The 32-bit operands are actually hurting code density most of the time. An instruction that would take 4 bytes in a RISC ISA takes at least 5 bytes in a CISC ISA. Same thing with the 32-bit offsets for x86_64 conditional branches. Edit: With "CISC style" I meant those instructions that use implicit registers instead of GPRs and/or do fancy multi-operation stuff, like LOOP or POPA/PUSHA etc.