4 ms·

> The amount of compute and/or data motion that can be achieved with a single instruction. This is really about amortizing the cost of decode and allowing the p

> The amount of compute and/or data motion that can be achieved with a single instruction. This is really about amortizing the cost of decode and allowing the pipeline to be kept full by producing a lot of work from a single instruction. I'm basing this on my knowledge from CS classes in the late 90s but isn't the point of RISC for the decode to be very simple because the instructions are very simple? Much of the optimization of x86 has been in the Decode where a CISC can be transformed in to multiple and often parallel instructions. And parallelism where we've seen our gains and benefited the most in the last ~10-15 years of CPU design. Does that mean in order for RISC to be performant some degree of parallelism will have to be done by the compiler? Isn't that what ultimately killed Itanium which was a VLIW processor?

seanmcdirmid 7y ago
Yes, even if VLIW failed, GPUs are showing us that there is a lot of performance to be had in switching from hardware to software scheduled parallel memory accesses.
- dragontamer 7y ago
  GPUs advantage isn't from scheduling IMO. You are correct in that GPUs have a degree of software scheduled memory instructions. There's no Out-of-order schedulers in AMD or NVidia. The best you got is "start memory load/store", and "synchronize" kind of instructions. On AMD, you perform s_waitcnt to wait for memory to respond. NVidia has read/write barriers + stall counts encoded into each instruction that performs a similar task. However, I argue that GPU advantage comes from the innate "warp" / "wavefront" model. The programmer will specify 100,000+ "SIMD threads" of execution. There are blatantly obvious ways to optimize the memory access when you have hundreds-of-thousands of threads stepping together. In particular: with 32-threads operating at a time, you can stripe the data so that the 32-threads all access 128-byte chunks at a time. (Thread-local variable "uint32_t foo" for threads #0 through #31 can be stored in memory location #0 through #127. When threads #0 through #31 read "foo" they only need to perform 2-memory transfers: because modern memory reads/writes 64-byte blocks at a time). "Striping" memory access, and grouping memory accesses together (aka coalescing) is very important for modern GPU (and CPU) memory optimization. Its far easier to do on GPUs due to the parallel model. In fact, all thread-local variables are optimized as such in GPU-code, its basically automatic. CPUs attempt to do the same through auto-prefetching. But it doesn't get as far as the GPU-approach. GPUs benefit from wide memory busses, because of the wavefront / warp model of programming. 32+ threads grab memory, and stall on memory, at the same time. Its more efficient to calculate memory stalls for the 32+ batch of threads (one warp) rather than doing it one-at-a-time like a CPU.
- Symmetry 7y ago
  GPUs are used in tasks where the memory access patterns are simple enough that you can predict them ahead of time. In cases where there isn't heaps of parallelism and memory access patterns are predictable people still use VLIW processors. Your smartphone probably has one.
dgacmu 7y ago
(1) That was part of the point of RISC. Other parts were that compiling for CISC was hard at the time. (2) The compiler is critical for performance on any platform these days. But RISC/CISC and VLIW vs out-of-order/speculative are separable. You can do an out-of-order speculative RISC machine and it does the same thing an x86 CPU does. (At the micro-op level they're pretty similar). For an OOO processor, the compiler still needs to do a decent-ish job of grouping instructions so that the processor can find the parallelism, because its execution window is limited -- the CPU won't look ahead 200 instructions to find more parallel stuff to execute. The requirements are even more stringent with an in-order VLIW core, where the compiler is responsible for all of it. I know of more production VLIW cores today than back in the Itanium days. Compilers have gotten better. The GPU manufacturers are an interesting case of this: AMD, for example, was VLIW until GCN in ~2012. NVidia tried VLIW in NV3x, but abandoned it in 2004-2005 with the move to more general-purpose computation on GPUs. Some of the AI hardware accelerators are looking again at VLIW now, probably because the regular, fixed things they do are more amenable to compiler-based scheduling, whereas very fine-grained dynamic scheduling is better done in hardware.
- wolf550e 7y ago
  > the CPU won't look ahead 200 instructions to find more parallel stuff to execute CPUs do: > Intel increased the reorder buffer by over 50% from 224 entries in Skylake to 352 entries in Sunny Cove. Likewise, the amount of inflight loads and stores has been increased by over 50% to 200 memory operations inflight. https://fuse.wikichip.org/news/2371/intel-sunny-cove-core-to-deliver-a-major-improvement-in-single-thread-performance-bigger-improvements-to-follow/ https://fuse.wikichip.org/news/2371/intel-sunny-cove-core-to...
  - that_jojo 7y ago
    The future is INSANE.
  - dgacmu 7y ago
    Thanks - rats, I had it in my head it was 192, but that was Haswell. The CPU won't look ahead 400 instructions! ;-)
Symmetry 7y ago
Ease of decode was one bit but there were a couple of other important ones. (1) Because RISC chips were simpler they consumed less transistors. Because there were less transistors you could fit an entire CPU on a single chip of silicon which drastically improved performance. (2) It was much easier to apply pipelining to load-store architectures than to architectures where instructions could contain multiple memory accesses or indirect memory accesses and also retain precise interrupts. Pipelining dramatically increased the rate at which instructions could execute. Also, back in the CISC days the limitations of the memory subsystems available made it supremely important to optimize for minimizing instruction size and instruction bandwidth. This was far less of a concern by the time the RISC paradigm rolled around compared to other issues.

seanmcdirmid 7y ago

Yes, even if VLIW failed, GPUs are showing us that there is a lot of performance to be had in switching from hardware to software scheduled parallel memory accesses.

dragontamer 7y ago
GPUs advantage isn't from scheduling IMO. You are correct in that GPUs have a degree of software scheduled memory instructions. There's no Out-of-order schedulers in AMD or NVidia. The best you got is "start memory load/store", and "synchronize" kind of instructions. On AMD, you perform s_waitcnt to wait for memory to respond. NVidia has read/write barriers + stall counts encoded into each instruction that performs a similar task. However, I argue that GPU advantage comes from the innate "warp" / "wavefront" model. The programmer will specify 100,000+ "SIMD threads" of execution. There are blatantly obvious ways to optimize the memory access when you have hundreds-of-thousands of threads stepping together. In particular: with 32-threads operating at a time, you can stripe the data so that the 32-threads all access 128-byte chunks at a time. (Thread-local variable "uint32_t foo" for threads #0 through #31 can be stored in memory location #0 through #127. When threads #0 through #31 read "foo" they only need to perform 2-memory transfers: because modern memory reads/writes 64-byte blocks at a time). "Striping" memory access, and grouping memory accesses together (aka coalescing) is very important for modern GPU (and CPU) memory optimization. Its far easier to do on GPUs due to the parallel model. In fact, all thread-local variables are optimized as such in GPU-code, its basically automatic. CPUs attempt to do the same through auto-prefetching. But it doesn't get as far as the GPU-approach. GPUs benefit from wide memory busses, because of the wavefront / warp model of programming. 32+ threads grab memory, and stall on memory, at the same time. Its more efficient to calculate memory stalls for the 32+ batch of threads (one warp) rather than doing it one-at-a-time like a CPU.
Symmetry 7y ago
GPUs are used in tasks where the memory access patterns are simple enough that you can predict them ahead of time. In cases where there isn't heaps of parallelism and memory access patterns are predictable people still use VLIW processors. Your smartphone probably has one.

dgacmu 7y ago

(1) That was part of the point of RISC. Other parts were that compiling for CISC was hard at the time. (2) The compiler is critical for performance on any platform these days. But RISC/CISC and VLIW vs out-of-order/speculative are separable. You can do an out-of-order speculative RISC machine and it does the same thing an x86 CPU does. (At the micro-op level they're pretty similar). For an OOO processor, the compiler still needs to do a decent-ish job of grouping instructions so that the processor can find the parallelism, because its execution window is limited -- the CPU won't look ahead 200 instructions to find more parallel stuff to execute. The requirements are even more stringent with an in-order VLIW core, where the compiler is responsible for all of it. I know of more production VLIW cores today than back in the Itanium days. Compilers have gotten better. The GPU manufacturers are an interesting case of this: AMD, for example, was VLIW until GCN in ~2012. NVidia tried VLIW in NV3x, but abandoned it in 2004-2005 with the move to more general-purpose computation on GPUs. Some of the AI hardware accelerators are looking again at VLIW now, probably because the regular, fixed things they do are more amenable to compiler-based scheduling, whereas very fine-grained dynamic scheduling is better done in hardware.

wolf550e 7y ago
> the CPU won't look ahead 200 instructions to find more parallel stuff to execute CPUs do: > Intel increased the reorder buffer by over 50% from 224 entries in Skylake to 352 entries in Sunny Cove. Likewise, the amount of inflight loads and stores has been increased by over 50% to 200 memory operations inflight. https://fuse.wikichip.org/news/2371/intel-sunny-cove-core-to-deliver-a-major-improvement-in-single-thread-performance-bigger-improvements-to-follow/ https://fuse.wikichip.org/news/2371/intel-sunny-cove-core-to...
- that_jojo 7y ago
  The future is INSANE.
- dgacmu 7y ago
  Thanks - rats, I had it in my head it was 192, but that was Haswell. The CPU won't look ahead 400 instructions! ;-)

Symmetry 7y ago

Ease of decode was one bit but there were a couple of other important ones. (1) Because RISC chips were simpler they consumed less transistors. Because there were less transistors you could fit an entire CPU on a single chip of silicon which drastically improved performance. (2) It was much easier to apply pipelining to load-store architectures than to architectures where instructions could contain multiple memory accesses or indirect memory accesses and also retain precise interrupts. Pipelining dramatically increased the rate at which instructions could execute. Also, back in the CISC days the limitations of the memory subsystems available made it supremely important to optimize for minimizing instruction size and instruction bandwidth. This was far less of a concern by the time the RISC paradigm rolled around compared to other issues.