8 ms·

The x86 decoders consume a reasonable amount of power, but the trouble is making them wider without affecting that. I have an AMD CPU. Zen CPUs come with a fai

by tux3 3y ago

The x86 decoders consume a reasonable amount of power, but the trouble is making them wider without affecting that. I have an AMD CPU. Zen CPUs come with a fairly wide backend. But the frontend is what it is (especially early Zen), and without SMT it's essentially impossible to keep all those execution units fed. It's not that 8 x86 decoders wouldn't be a benefit, it's just that more decoders isn't cheap in x86 cores, each extra decoder is a serious cost. If you compare with the big ARM cores, having a wide frontend is not a complex research problem or an impractical cost. 8 wide ARM decode is completely practical. You even have open source superscalar RISC-V cores just publicly available on Github running on FPGAs with 8 wide decode. Large frontends are (relatively) cheap and easy, if you're not x86. So when we notice that the narrower x86 CPU's decode doesn't consume that much (a "drop in the ocean"), that's because it was designed narrower to keep the PPA reasonable! The reason I can't feed my Zen backend isn't because having a wide frontend is useless and I should just enable SMT anyways, it's because x86 makes wide decodes much less practical than competing architectures.

ip26 3y ago
It's a trade-off between problems. Variable length instructions are not as trivial to decode wide, so you need more cleverness here. However, fixed length instructions decrease code density, which asks more of the instruction cache. Note Zen4 has a 32 KB L1 instruction cache while the M1 has a 192 KB L1 instruction cache, requiring extra cleverness here instead to handle the higher latency and area. Meanwhile, micro-op caches hide both problems. The are ripple effects to consider as well. The large L1 caches of M1 (320 KB total) put capacity pressure on L2, towards larger sizes and/or away from inclusive policy. See the 12MB shared L2. Meanwhile, the narrower decode of Zen4 puts pressure on things like branch prediction accuracy & mispredict correction latency - if you predicted the wrong codepath, you can't catch up as quickly. See the large branch predictors on Zen4.
- pclmulqdq 3y ago
  ARM used to find itself on the wrong side of this tradeoff in the era of 4-wide x86 decode units and 4-6 wide ARM decoders. They lost too much perf to cache size for the decoder width to make up for it. It's unclear to me if they will pull ahead on the perf/area game with the era of 8-wide x86 decoders coming.
- cesarb 3y ago
  > Note Zen4 has a 32 KB L1 instruction cache while the M1 has a 192 KB L1 instruction cache, requiring extra cleverness here instead to handle the higher latency and area. There's another factor here: to have a low latency, the L1 cache has to be indexed by the bits which don't change when translating from virtual addresses to physical addresses. That makes it harder to have a larger low-latency L1 cache when the native page size is 4KiB (AMD/Intel) instead of 16KiB (Apple M1/M2). That is, most of the "cleverness" allowing for a larger L1 instruction cache is simply a larger page size.
  - mjevans 3y ago
    If the instructions are on average 3-4x less dense (take about that much more space) than the trade-off in associativity granularity and corresponding increase in cache size are logical. The logical management circuits would be around the same size, though the number of memory cells and corresponding cost in silicon, power / thermal, and signal propagation / layout issues remain.
  - rep_lodsb 3y ago
    Since its instructions have to be aligned, ARM would also have only 12 usable address bits per page.
  - brigade 3y ago
    Pretty sure everyone uses a VIPT L1, what evidence is there that it's PIPT? (also what do you believe happens with 4k pages on M1, since it does support those, or why did the A7 have larger caches than Zen4 a decade ago, which was well before 16k pages)
- codedokode 3y ago
  x86 also uses 2-address instructions which means that you often need to use moves between registers (additional instructions), example: [1]. ARM uses 3-address instructions. Also, x86 code is compact, but not as compact as in era of 8080 [2] - here addition and multiplication require 3 bytes each, 6 bytes total. To my surprise, ARM has an add-multiply instruction and it uses just 4 bytes (instead of 8) [3]. And RISC-V uses 6 bytes because of shortened instruction for addition [4] Of course, this simple function cannot be a replacement for proper analysis, but it seems that x86 code is not significantly denser. Also to my great disappoitment none of those CPUs has checked overflow for arithmetic operation. [1] https://godbolt.org/z/jsoccE5jv https://godbolt.org/z/jsoccE5jv [2] https://godbolt.org/z/jTMs1MEzh https://godbolt.org/z/jTMs1MEzh [3] https://godbolt.org/z/nGb8qKcxe https://godbolt.org/z/nGb8qKcxe [4] https://godbolt.org/z/x9c115crY https://godbolt.org/z/x9c115crY
  - BeeOnRope 3y ago
    x86-64 is not even a particularly compact encoding even compared to contemporary fixed length encodings. The inherent advantage of variable length encoding is largely cancelled out by wasted encoding space for legacy cruft. Aarch64 is roughly on par for encoding density.
  - kazinator 3y ago
    It's incredible how large the code is when you compile some trivial piece of C with GCC for x86 or amd64, compared to one's memory of working with C on 386 boxes. What? I just wrote a simple main and a couple of small helper functions. How can we be past 20 kB of .text section?
    - throwaway2037 3y ago
      Can you share a sample with Godbolt? It might foster some good discussion here.
  - userbinator 3y ago
    Of course, this simple function cannot be a replacement for proper analysis, but it seems that x86 code is not significantly denser. Look at the demoscene for an example of dense x86 code, especially in the sub-1K categories. They routinely achieve code densities for x86 that no compilers I know of can get close to, and AFAIK I have not seen the same happen with ARM, nor MIPS or any other well-known RISC. 3-address instructions improve code density only in situations where both source operands are needed later.
- crest 3y ago
  The M1 uses 16kiB pages to reduce TLB pressure and get more non-aliasing cache tag bits.
  - sliken 3y ago
    Can use 16k pages. M1 also can use 4k pages.
- audunw 3y ago
  > However, fixed length instructions decrease code density, which asks more of the instruction cache. The problem is that x86 has a lot of legacy that wastes the potential gain of variable length encoding. If I remember correctly, the code density of 64-bit x86 isn't really significantly better than 64-bit ARM, despite ARM dropping support for Thumb. RISC-V is going to be interesting in that regard because with the latest revisions of the compressed instruction format, it can potentially beat both ARM and x86 on real-world code density, while still being easier to decode.
- nayuki 3y ago
  > Variable length instructions are not as trivial to decode wide, so you need more cleverness here. However, fixed length instructions decrease code density This is true in theory but I question how much it applies to current CPUs. A problem I see is that the x86 instruction set was designed in the 1970s with corresponding assumptions about instruction frequency, speed of memory, number of CPU cycles, pipelining (or lack thereof), addressing modes, etc. Even if the variable-length encoding was close to optimal back then, there's no way it's remotely optimal given all the extensions to the instruction set, the change from hand-coded assembly to compiler-generated code, and many more seismic shifts in the ISA. So I think x86's variable-length instructions now have all the downsides of complicated decoding as well as the downsides of suboptimal code density. But you also can't reshuffle all the instructions and addressing modes without breaking backward compatibility.
phkahler 3y ago
>> I have an AMD CPU. Zen CPUs come with a fairly wide backend. But the frontend is what it is... Zen 5 is widening the front end. My guess is with scaling coming to an end, one nice tweak with Zen 6 should be darn near the end of the performance road for a bit. Not saying the actual end, but it should be one of those sweet spots where you build a PC and it's really good for years to come. I'm still running Raven Ridge and have no need to upgrade, but I will when I can get double the cores or more at double the IPC or more, and maybe at lower power ;-)
tester756 3y ago
>The x86 decoders consume a reasonable amount of power this article states otherwise, so how is it?
- LegionMammal978 3y ago
  I think the intent here is to say that decoders' current power consumption is reasonable (i.e., within reason), but widening them could make their power consumption unreasonable (i.e., too high).
  - TinkersW 3y ago
    Maybe but maybe not.. the E cores on Raptorlake have two 3 wide decoders, apparently it can decode 6 instructions this way(though how often or reliably that is I don't think Intel has said).
  - clamchowder 3y ago
    Note that Golden Cove has a 6-wide decoder, and does not have a clear advantage over Zen 4 with a 4-wide decoder. Other parts of the architecture affect performance more, so it doesn't make sense to put more area and power budget into the decoder. (also I'm the author of the article)
BeeOnRope 3y ago
Having a wide backend which cannot be fully utilized is common in non-x86 chips as well. The backend units are specialized while the front end less so, so to sustain the front-end bandwidth for many instruction mixed you need a wider backend so you have enough units of the right type. Wide x86 decode definitely has a cost, but I don't think it's the primary limiter as you make it out to be: for several x86 generations the narrowest bottleneck has been the renamer which has crept up from 3 to 6 in a glacial manner. Admittedly rename is also complicated on x86 due to memory source/destinations, 3 input instructions, 2 output instructions, etc.
bigdict 3y ago
Is that because of ISA complexity? Especially with x86 growth by accretion while keeping backward compatibility all the way to 16-bit. So you could conceivably afford a wide frontend if you restricted x86 to a subset (64 bit only, drop a bunch of weird CISC'y instructions).
- amluto 3y ago
  It’s mainly (I think) the distribution of instruction lengths. x86 instructions have any length from 1 to 15 bytes. A decoder wants to decode multiple instructions per cycle, and it generally does this in parallel, by simultaneously deciding at multiple starting points. With a fixed length ISA, to decode n instructions, you just decode them. With x86, if you simultaneously decode at offset 0, 1, …, 7, you have 8 decoders but are only likely to decode a couple of correct instructions. The rest start in the middle of an instruction and need to be discarded. So you either need many more parallel decoders for the same throughput or a more complex system to try to avoid throwing away so much work.
  - jabl 3y ago
    > ... or a more complex system to try to avoid throwing away so much work. IIRC Jim Keller said in some interview that modern x86 uses prediction and speculation (similar to branch prediction), and it works surprisingly well.
    - amluto 3y ago
      I’m sure this is doable, but I would certainly count it as “complex”. But fundamentally, a given chip, dedicating a given area to the task, can only begin to decode at so many positions per cycle. And the more intelligent it tries to be about where to start decoding, the longer into that cycle it needs to wait. And one nastiness about x86 is that you have to decode pretty far into an instruction to even determine its likely length. You can’t do something like looking up the likely length in a table indexed by the first byte of an instruction. I wonder whether modern chips have pipelined decoders.
adfgionionio 3y ago
The decoders are largely irrelevant. All modern x86 machines use a uop cache. The large majority of instructions hit in this cache and do not need to hit the decoders at all. As a result, the decoders can spend much of their time shut down. You already have four-ish decoders that are idle most of the time; why do you want eight-ish decoders that are idle nearly all the time? No one would design an ISA like x86 these days. It definitely does use more power and more die area than strictly needed. It definitely does reduce performance in some applications. It definitely did take heroic engineering efforts to make x86 work well. But, all told, it just doesn't matter very much.
- CalChris 3y ago
  The hit rate on that UOP cache is only about 80%, according to Intel. And UOP cache misses will have strong locality, so they can’t really be hidden.
Stratoscope 3y ago
Acronym/initialism definer here. There are two in your comment that I wasn't familiar with, although I understood them once I saw them spelled out. SMT is simultaneous multithreading, which may be more familiar under Intel's name of hyper-threading. https://en.wikipedia.org/wiki/Simultaneous_multithreading https://en.wikipedia.org/wiki/Simultaneous_multithreading PPA is not the Professional Photographers of America, nor the Professional Pickleball Association, nor the Philadelphia Parking Authority, all of which came up at the top of my naïve search. It's Power-Performance-Area. https://en.wikichip.org/wiki/power-performance-area https://en.wikichip.org/wiki/power-performance-area
- akvadrako 3y ago
  SMT is not just hyper-threading, it also includes having multiple CPUs, including sockets and cores.
  - gpderetta 3y ago
    No, that's SMP. SMT is specifically multiple instruction pointers (threads) on a single core, i.e Hyperthreading
    - dist1ll 3y ago
      (Warning: super pedantic) Actually, what akvadrako described is just called multiprocessing - not SMP. SMP refers specifically to systems with uniform latency to shared memory. So any distributed memory topology like ccNUMA w/ interconnect would not fit that definition.
      - gpderetta 3y ago
        Very good point NUMA != SMP. I Didn't catch that difference before.
    - quincunx 3y ago
      "No, that's SMP" - is perhaps too resolute a definition. I'd say SMP is across processes, SMT is within a process, which would put the distinction in software, not hardware architecture. Definitions are more grey, muddy and dependent on context; in that line of thought, I feel the parent you're responding to is accurate enough to not warrant criticism or even downvotes.
sweetjuly 3y ago
>You even have open source superscalar RISC-V cores just publicly available on Github running on FPGAs with 8 wide decode. Large frontends are (relatively) cheap and easy, if you're not x86. Which one? I know BOOM can technically go eight wide in so far as it's parametrizable but I suspect any BOOM backend which could support that much throughout would be a nightmare to instantiate on nearly any FPGA.
- tux3 3y ago
  I had VROOM! in mind (https://github.com/MoonbaseOtago/vroom https://github.com/MoonbaseOtago/vroom) because I remembered it aims for 4 IPC avg with a width of 8. Though looking again it's 8 compressed 16 bit instructions or 4 uncompressed 32 bit instruction. So you could argue a real mix of instructions is not going to be all 16 bit but some 16 and some 32, so the 8 is rarely achieved in practice, and also the block diagram only shows 4 decode blocks. But it can in fact peak at 8 instructions decoded per clock, so I'll call that 8 wide decode. (You could even argue it's especially impressive, since RISC-V technically qualifies as variable-length encoding like x86, it's just that only the 16/32 instructions encoding are really in use at the moment)
  - Taniwha 3y ago
    As you point out current VROOM! is 4-8 into the decoder depending on the mix of incoming instructions, each decode block can decode one 32-bit instruction, or 2 16-bit ones. A VERY general rule of thumb is that every 5th instruction is a branch, decoding more than 8 may be pointless However VROOM! can replay 8 instructions per clock out of the trace cache .... (they're already decoded and get pushed into the pipe after the decoders)
    - sweetjuly 3y ago
      Hey! I looked at your blog and loved your bug analysis, it's extremely rare to see this stuff publicly but I'm excited for more as the FOSS CPU scene speeds up. Hopefully the wide decode will be more useful when trace cache lands. Is VROOM primarily targeting ASICs of FPGAs? I know BOOM is primarily meant for ASICs but I feel like there is still a lot of room to build a good OoO FPGA core, particularly since us mere mortals can't get a spot on a wafer big enough to support even the smallest configs of BOOM (or presumably VROOM) since the efabless Sky130 shuttles, while "affordable", are hardly big enough to most OoO cores. I've been thinking about taking the ideas from this paper [1] and adding it to BOOM to try and make it more FPGA friendly since that seems like the most realistic way to get a very high performance custom core at this point. [1] https://ieeexplore.ieee.org/document/8977924/ https://ieeexplore.ieee.org/document/8977924/
13of40 3y ago
I'm sure someone has thought of this before, but there's x86 as the full ISA, and there's x86 as gcc (or whatever) actually uses it. What if you had one or two cores that actually run x86 and the rest just run a reasonable subset of it until they hit a weird instruction and hand the thread over to the more formal one?
- StressedDev 3y ago
  You don't need special cores. One thing a lot of people do not seem to realize is compilers do not use the full instruction set. They use a subset and this subset is what has to be fast. Here are the common x86 instructions I remember when I debugged disassembled source code (1). - mov (an instruction to load, store data) - Basic arithmetic instructions (add, sub, div, mul, etc.) - Bitwise operators (or, and, xor, shift, etc.) - Compare (cmp) - Branch (jz, jnz, j, ret, etc.) - There was one weird instruction I would occasionally see. It was used to zero memory (I forget its name). My guess is the compiler used it because it was faster than running creating a loop which zeroed memory. The point is, people seem to assume ALL instructions have to be fast. The only instructions which have to be fast are the ones the compilers are using. The rest just have to work. Making the rarely used instructions work probably takes up a negligible amount of space on a typical core. Moving these instructions to a special core would not get you much but would make old software run slower (becuase of the context switch to the other core and/or a limited number of cores it could run on). Note that I know AMD and Intel both have a lot of guidance for compiler writers. (1) I debugged disabled code because I was debugging optimized code and it was the only way to reliable determine why a function crashed or malfunctioned.
  - Someone 3y ago
    > One thing a lot of people do not seem to realize is compilers do not use the full instruction set. They use a subset and this subset is what has to be fast. That’s a self-fulfilling prophecy. Compiler writers won’t use the instructions that are (relatively) slow. If, tomorrow, Intel makes ‘add’ slow, compilers will eventually start compiling addition as negation + subtraction (and AMD will become more popular) That’s what makes hardware designers unhappy sometimes. If they ship a shiny faster CPU that requires a large overhaul of compilers, it can take time for compiler writers to catch up. Hardware designers shouldn’t (only) aim for getting closer to the local optimum of where compiler writers are, but also for the higher peaks of where they could be.
fulafel 3y ago
Do you have references about AMD CPUs being more frequently bottlenecked by instruction decode despite uop caches (vs non x86)?
- tux3 3y ago
  I don't have any good benchmark to link to off the top of my head, but I'll handwave in the general direction of Agner Fog's guide, which is accurate in my experience (and generally a great resource): https://www.agner.org/optimize/microarchitecture.pdf https://www.agner.org/optimize/microarchitecture.pdf From the multiple "Bottlenecks in AMD Zen" sections, a common point is >The limiting fetch rate of up to 16 bytes per clock is a very likely bottleneck for CPU- intensive code with large loops Although admittedly, if your small hot loops fit in the uop cache that does largely mitigate the fetch/decode problem
sylware 3y ago
Performant risc-v implementations for desktop/mobile/server, fast!
- snvzz 3y ago
  How fast do you need them to be? e.g. we know about Ventana Veyron, TBA before end of year, and Tenstorrent's Ascalon, TBA 2024, competitive with Zen5 which is also 2024. Ascalon is 8-wide, but has smaller siblings at lesser decoder width, to cover a range of uses. There's also Rivos, MIPS and SiFive working on very high performance cores, but we know less about these efforts.
  - sylware 3y ago
    On desktop, fast enough to run state-of-the-art games, decode 4k av1/h265 videos at 60fps (thread performance) without a 1kW power supply and water cooling. Since there will be a transition period before heavy assembly optimization starts to kick in (similar to dav1d decoder), would start with gcc/clang generated machine code. Oh, and should compile a full blown linux kernel and those horrible gcc/llvm as fast as their arm/x86_64 counter-parts. Compression/decompressing would be a good benchmark too. Mobile: ratio power consumption/performance to run as smooth as the latest samsung galaxy flag ship the CPU-wise worst android apps, off AC. Server: erf, I guess that would be tons of cores with tons of ram and cache then back to full blown linux kernel/gcc/llvm compilation speed&compression/decompressing&etc. We are talking about world domination of a worldwide royalty free/ultra stable in time ISA (oxymoron).
    - snvzz 3y ago
      Your post used "performant" and "fast". I understood fast as to do with roadmaps, and used it that way. Before year end for Veyron, next year for Ascalon. Potentially, there'll be further announcements for performant microarchitectures next year. As for performance, with Ascalon promising similar to Zen5 performance at significantly reduced power consumption, I don't think there's much reason for concern: Very performant RISC-V microarchitectures are coming. With all these companies and their strong teams working on new RISC-V microarchitectures, there'll be plenty of choice, soon.

by tux3 3y ago

ip26 3y ago

It's a trade-off between problems. Variable length instructions are not as trivial to decode wide, so you need more cleverness here. However, fixed length instructions decrease code density, which asks more of the instruction cache. Note Zen4 has a 32 KB L1 instruction cache while the M1 has a 192 KB L1 instruction cache, requiring extra cleverness here instead to handle the higher latency and area. Meanwhile, micro-op caches hide both problems. The are ripple effects to consider as well. The large L1 caches of M1 (320 KB total) put capacity pressure on L2, towards larger sizes and/or away from inclusive policy. See the 12MB shared L2. Meanwhile, the narrower decode of Zen4 puts pressure on things like branch prediction accuracy & mispredict correction latency - if you predicted the wrong codepath, you can't catch up as quickly. See the large branch predictors on Zen4.

pclmulqdq 3y ago
ARM used to find itself on the wrong side of this tradeoff in the era of 4-wide x86 decode units and 4-6 wide ARM decoders. They lost too much perf to cache size for the decoder width to make up for it. It's unclear to me if they will pull ahead on the perf/area game with the era of 8-wide x86 decoders coming.
cesarb 3y ago
> Note Zen4 has a 32 KB L1 instruction cache while the M1 has a 192 KB L1 instruction cache, requiring extra cleverness here instead to handle the higher latency and area. There's another factor here: to have a low latency, the L1 cache has to be indexed by the bits which don't change when translating from virtual addresses to physical addresses. That makes it harder to have a larger low-latency L1 cache when the native page size is 4KiB (AMD/Intel) instead of 16KiB (Apple M1/M2). That is, most of the "cleverness" allowing for a larger L1 instruction cache is simply a larger page size.
- mjevans 3y ago
  If the instructions are on average 3-4x less dense (take about that much more space) than the trade-off in associativity granularity and corresponding increase in cache size are logical. The logical management circuits would be around the same size, though the number of memory cells and corresponding cost in silicon, power / thermal, and signal propagation / layout issues remain.
- rep_lodsb 3y ago
  Since its instructions have to be aligned, ARM would also have only 12 usable address bits per page.
- brigade 3y ago
  Pretty sure everyone uses a VIPT L1, what evidence is there that it's PIPT? (also what do you believe happens with 4k pages on M1, since it does support those, or why did the A7 have larger caches than Zen4 a decade ago, which was well before 16k pages)
codedokode 3y ago
x86 also uses 2-address instructions which means that you often need to use moves between registers (additional instructions), example: [1]. ARM uses 3-address instructions. Also, x86 code is compact, but not as compact as in era of 8080 [2] - here addition and multiplication require 3 bytes each, 6 bytes total. To my surprise, ARM has an add-multiply instruction and it uses just 4 bytes (instead of 8) [3]. And RISC-V uses 6 bytes because of shortened instruction for addition [4] Of course, this simple function cannot be a replacement for proper analysis, but it seems that x86 code is not significantly denser. Also to my great disappoitment none of those CPUs has checked overflow for arithmetic operation. [1] https://godbolt.org/z/jsoccE5jv https://godbolt.org/z/jsoccE5jv [2] https://godbolt.org/z/jTMs1MEzh https://godbolt.org/z/jTMs1MEzh [3] https://godbolt.org/z/nGb8qKcxe https://godbolt.org/z/nGb8qKcxe [4] https://godbolt.org/z/x9c115crY https://godbolt.org/z/x9c115crY
- BeeOnRope 3y ago
  x86-64 is not even a particularly compact encoding even compared to contemporary fixed length encodings. The inherent advantage of variable length encoding is largely cancelled out by wasted encoding space for legacy cruft. Aarch64 is roughly on par for encoding density.
- kazinator 3y ago
  It's incredible how large the code is when you compile some trivial piece of C with GCC for x86 or amd64, compared to one's memory of working with C on 386 boxes. What? I just wrote a simple main and a couple of small helper functions. How can we be past 20 kB of .text section?
  - throwaway2037 3y ago
    Can you share a sample with Godbolt? It might foster some good discussion here.
- userbinator 3y ago
  Of course, this simple function cannot be a replacement for proper analysis, but it seems that x86 code is not significantly denser. Look at the demoscene for an example of dense x86 code, especially in the sub-1K categories. They routinely achieve code densities for x86 that no compilers I know of can get close to, and AFAIK I have not seen the same happen with ARM, nor MIPS or any other well-known RISC. 3-address instructions improve code density only in situations where both source operands are needed later.
crest 3y ago
The M1 uses 16kiB pages to reduce TLB pressure and get more non-aliasing cache tag bits.
- sliken 3y ago
  Can use 16k pages. M1 also can use 4k pages.
audunw 3y ago
> However, fixed length instructions decrease code density, which asks more of the instruction cache. The problem is that x86 has a lot of legacy that wastes the potential gain of variable length encoding. If I remember correctly, the code density of 64-bit x86 isn't really significantly better than 64-bit ARM, despite ARM dropping support for Thumb. RISC-V is going to be interesting in that regard because with the latest revisions of the compressed instruction format, it can potentially beat both ARM and x86 on real-world code density, while still being easier to decode.
nayuki 3y ago
> Variable length instructions are not as trivial to decode wide, so you need more cleverness here. However, fixed length instructions decrease code density This is true in theory but I question how much it applies to current CPUs. A problem I see is that the x86 instruction set was designed in the 1970s with corresponding assumptions about instruction frequency, speed of memory, number of CPU cycles, pipelining (or lack thereof), addressing modes, etc. Even if the variable-length encoding was close to optimal back then, there's no way it's remotely optimal given all the extensions to the instruction set, the change from hand-coded assembly to compiler-generated code, and many more seismic shifts in the ISA. So I think x86's variable-length instructions now have all the downsides of complicated decoding as well as the downsides of suboptimal code density. But you also can't reshuffle all the instructions and addressing modes without breaking backward compatibility.

phkahler 3y ago

>> I have an AMD CPU. Zen CPUs come with a fairly wide backend. But the frontend is what it is... Zen 5 is widening the front end. My guess is with scaling coming to an end, one nice tweak with Zen 6 should be darn near the end of the performance road for a bit. Not saying the actual end, but it should be one of those sweet spots where you build a PC and it's really good for years to come. I'm still running Raven Ridge and have no need to upgrade, but I will when I can get double the cores or more at double the IPC or more, and maybe at lower power ;-)

tester756 3y ago

>The x86 decoders consume a reasonable amount of power this article states otherwise, so how is it?

LegionMammal978 3y ago
I think the intent here is to say that decoders' current power consumption is reasonable (i.e., within reason), but widening them could make their power consumption unreasonable (i.e., too high).
- TinkersW 3y ago
  Maybe but maybe not.. the E cores on Raptorlake have two 3 wide decoders, apparently it can decode 6 instructions this way(though how often or reliably that is I don't think Intel has said).
- clamchowder 3y ago
  Note that Golden Cove has a 6-wide decoder, and does not have a clear advantage over Zen 4 with a 4-wide decoder. Other parts of the architecture affect performance more, so it doesn't make sense to put more area and power budget into the decoder. (also I'm the author of the article)

BeeOnRope 3y ago

Having a wide backend which cannot be fully utilized is common in non-x86 chips as well. The backend units are specialized while the front end less so, so to sustain the front-end bandwidth for many instruction mixed you need a wider backend so you have enough units of the right type. Wide x86 decode definitely has a cost, but I don't think it's the primary limiter as you make it out to be: for several x86 generations the narrowest bottleneck has been the renamer which has crept up from 3 to 6 in a glacial manner. Admittedly rename is also complicated on x86 due to memory source/destinations, 3 input instructions, 2 output instructions, etc.

bigdict 3y ago

Is that because of ISA complexity? Especially with x86 growth by accretion while keeping backward compatibility all the way to 16-bit. So you could conceivably afford a wide frontend if you restricted x86 to a subset (64 bit only, drop a bunch of weird CISC'y instructions).

amluto 3y ago
It’s mainly (I think) the distribution of instruction lengths. x86 instructions have any length from 1 to 15 bytes. A decoder wants to decode multiple instructions per cycle, and it generally does this in parallel, by simultaneously deciding at multiple starting points. With a fixed length ISA, to decode n instructions, you just decode them. With x86, if you simultaneously decode at offset 0, 1, …, 7, you have 8 decoders but are only likely to decode a couple of correct instructions. The rest start in the middle of an instruction and need to be discarded. So you either need many more parallel decoders for the same throughput or a more complex system to try to avoid throwing away so much work.
- jabl 3y ago
  > ... or a more complex system to try to avoid throwing away so much work. IIRC Jim Keller said in some interview that modern x86 uses prediction and speculation (similar to branch prediction), and it works surprisingly well.
  - amluto 3y ago
    I’m sure this is doable, but I would certainly count it as “complex”. But fundamentally, a given chip, dedicating a given area to the task, can only begin to decode at so many positions per cycle. And the more intelligent it tries to be about where to start decoding, the longer into that cycle it needs to wait. And one nastiness about x86 is that you have to decode pretty far into an instruction to even determine its likely length. You can’t do something like looking up the likely length in a table indexed by the first byte of an instruction. I wonder whether modern chips have pipelined decoders.

adfgionionio 3y ago

The decoders are largely irrelevant. All modern x86 machines use a uop cache. The large majority of instructions hit in this cache and do not need to hit the decoders at all. As a result, the decoders can spend much of their time shut down. You already have four-ish decoders that are idle most of the time; why do you want eight-ish decoders that are idle nearly all the time? No one would design an ISA like x86 these days. It definitely does use more power and more die area than strictly needed. It definitely does reduce performance in some applications. It definitely did take heroic engineering efforts to make x86 work well. But, all told, it just doesn't matter very much.

CalChris 3y ago
The hit rate on that UOP cache is only about 80%, according to Intel. And UOP cache misses will have strong locality, so they can’t really be hidden.

Stratoscope 3y ago

Acronym/initialism definer here. There are two in your comment that I wasn't familiar with, although I understood them once I saw them spelled out. SMT is simultaneous multithreading, which may be more familiar under Intel's name of hyper-threading. https://en.wikipedia.org/wiki/Simultaneous_multithreading https://en.wikipedia.org/wiki/Simultaneous_multithreading PPA is not the Professional Photographers of America, nor the Professional Pickleball Association, nor the Philadelphia Parking Authority, all of which came up at the top of my naïve search. It's Power-Performance-Area. https://en.wikichip.org/wiki/power-performance-area https://en.wikichip.org/wiki/power-performance-area

akvadrako 3y ago
SMT is not just hyper-threading, it also includes having multiple CPUs, including sockets and cores.
- gpderetta 3y ago
  No, that's SMP. SMT is specifically multiple instruction pointers (threads) on a single core, i.e Hyperthreading
  - dist1ll 3y ago
    (Warning: super pedantic) Actually, what akvadrako described is just called multiprocessing - not SMP. SMP refers specifically to systems with uniform latency to shared memory. So any distributed memory topology like ccNUMA w/ interconnect would not fit that definition.
    - gpderetta 3y ago
      Very good point NUMA != SMP. I Didn't catch that difference before.
  - quincunx 3y ago
    "No, that's SMP" - is perhaps too resolute a definition. I'd say SMP is across processes, SMT is within a process, which would put the distinction in software, not hardware architecture. Definitions are more grey, muddy and dependent on context; in that line of thought, I feel the parent you're responding to is accurate enough to not warrant criticism or even downvotes.

sweetjuly 3y ago

>You even have open source superscalar RISC-V cores just publicly available on Github running on FPGAs with 8 wide decode. Large frontends are (relatively) cheap and easy, if you're not x86. Which one? I know BOOM can technically go eight wide in so far as it's parametrizable but I suspect any BOOM backend which could support that much throughout would be a nightmare to instantiate on nearly any FPGA.

tux3 3y ago
I had VROOM! in mind (https://github.com/MoonbaseOtago/vroom https://github.com/MoonbaseOtago/vroom) because I remembered it aims for 4 IPC avg with a width of 8. Though looking again it's 8 compressed 16 bit instructions or 4 uncompressed 32 bit instruction. So you could argue a real mix of instructions is not going to be all 16 bit but some 16 and some 32, so the 8 is rarely achieved in practice, and also the block diagram only shows 4 decode blocks. But it can in fact peak at 8 instructions decoded per clock, so I'll call that 8 wide decode. (You could even argue it's especially impressive, since RISC-V technically qualifies as variable-length encoding like x86, it's just that only the 16/32 instructions encoding are really in use at the moment)
- Taniwha 3y ago
  As you point out current VROOM! is 4-8 into the decoder depending on the mix of incoming instructions, each decode block can decode one 32-bit instruction, or 2 16-bit ones. A VERY general rule of thumb is that every 5th instruction is a branch, decoding more than 8 may be pointless However VROOM! can replay 8 instructions per clock out of the trace cache .... (they're already decoded and get pushed into the pipe after the decoders)
  - sweetjuly 3y ago
    Hey! I looked at your blog and loved your bug analysis, it's extremely rare to see this stuff publicly but I'm excited for more as the FOSS CPU scene speeds up. Hopefully the wide decode will be more useful when trace cache lands. Is VROOM primarily targeting ASICs of FPGAs? I know BOOM is primarily meant for ASICs but I feel like there is still a lot of room to build a good OoO FPGA core, particularly since us mere mortals can't get a spot on a wafer big enough to support even the smallest configs of BOOM (or presumably VROOM) since the efabless Sky130 shuttles, while "affordable", are hardly big enough to most OoO cores. I've been thinking about taking the ideas from this paper [1] and adding it to BOOM to try and make it more FPGA friendly since that seems like the most realistic way to get a very high performance custom core at this point. [1] https://ieeexplore.ieee.org/document/8977924/ https://ieeexplore.ieee.org/document/8977924/

13of40 3y ago

I'm sure someone has thought of this before, but there's x86 as the full ISA, and there's x86 as gcc (or whatever) actually uses it. What if you had one or two cores that actually run x86 and the rest just run a reasonable subset of it until they hit a weird instruction and hand the thread over to the more formal one?

StressedDev 3y ago
You don't need special cores. One thing a lot of people do not seem to realize is compilers do not use the full instruction set. They use a subset and this subset is what has to be fast. Here are the common x86 instructions I remember when I debugged disassembled source code (1). - mov (an instruction to load, store data) - Basic arithmetic instructions (add, sub, div, mul, etc.) - Bitwise operators (or, and, xor, shift, etc.) - Compare (cmp) - Branch (jz, jnz, j, ret, etc.) - There was one weird instruction I would occasionally see. It was used to zero memory (I forget its name). My guess is the compiler used it because it was faster than running creating a loop which zeroed memory. The point is, people seem to assume ALL instructions have to be fast. The only instructions which have to be fast are the ones the compilers are using. The rest just have to work. Making the rarely used instructions work probably takes up a negligible amount of space on a typical core. Moving these instructions to a special core would not get you much but would make old software run slower (becuase of the context switch to the other core and/or a limited number of cores it could run on). Note that I know AMD and Intel both have a lot of guidance for compiler writers. (1) I debugged disabled code because I was debugging optimized code and it was the only way to reliable determine why a function crashed or malfunctioned.
- Someone 3y ago
  > One thing a lot of people do not seem to realize is compilers do not use the full instruction set. They use a subset and this subset is what has to be fast. That’s a self-fulfilling prophecy. Compiler writers won’t use the instructions that are (relatively) slow. If, tomorrow, Intel makes ‘add’ slow, compilers will eventually start compiling addition as negation + subtraction (and AMD will become more popular) That’s what makes hardware designers unhappy sometimes. If they ship a shiny faster CPU that requires a large overhaul of compilers, it can take time for compiler writers to catch up. Hardware designers shouldn’t (only) aim for getting closer to the local optimum of where compiler writers are, but also for the higher peaks of where they could be.

fulafel 3y ago

Do you have references about AMD CPUs being more frequently bottlenecked by instruction decode despite uop caches (vs non x86)?

tux3 3y ago
I don't have any good benchmark to link to off the top of my head, but I'll handwave in the general direction of Agner Fog's guide, which is accurate in my experience (and generally a great resource): https://www.agner.org/optimize/microarchitecture.pdf https://www.agner.org/optimize/microarchitecture.pdf From the multiple "Bottlenecks in AMD Zen" sections, a common point is >The limiting fetch rate of up to 16 bytes per clock is a very likely bottleneck for CPU- intensive code with large loops Although admittedly, if your small hot loops fit in the uop cache that does largely mitigate the fetch/decode problem

sylware 3y ago

Performant risc-v implementations for desktop/mobile/server, fast!

snvzz 3y ago
How fast do you need them to be? e.g. we know about Ventana Veyron, TBA before end of year, and Tenstorrent's Ascalon, TBA 2024, competitive with Zen5 which is also 2024. Ascalon is 8-wide, but has smaller siblings at lesser decoder width, to cover a range of uses. There's also Rivos, MIPS and SiFive working on very high performance cores, but we know less about these efforts.
- sylware 3y ago
  On desktop, fast enough to run state-of-the-art games, decode 4k av1/h265 videos at 60fps (thread performance) without a 1kW power supply and water cooling. Since there will be a transition period before heavy assembly optimization starts to kick in (similar to dav1d decoder), would start with gcc/clang generated machine code. Oh, and should compile a full blown linux kernel and those horrible gcc/llvm as fast as their arm/x86_64 counter-parts. Compression/decompressing would be a good benchmark too. Mobile: ratio power consumption/performance to run as smooth as the latest samsung galaxy flag ship the CPU-wise worst android apps, off AC. Server: erf, I guess that would be tons of cores with tons of ram and cache then back to full blown linux kernel/gcc/llvm compilation speed&compression/decompressing&etc. We are talking about world domination of a worldwide royalty free/ultra stable in time ISA (oxymoron).
  - snvzz 3y ago
    Your post used "performant" and "fast". I understood fast as to do with roadmaps, and used it that way. Before year end for Veyron, next year for Ascalon. Potentially, there'll be further announcements for performant microarchitectures next year. As for performance, with Ascalon promising similar to Zen5 performance at significantly reduced power consumption, I don't think there's much reason for concern: Very performant RISC-V microarchitectures are coming. With all these companies and their strong teams working on new RISC-V microarchitectures, there'll be plenty of choice, soon.