4 ms·

How wasteful is it to have a bimodal decoder that assumes 8 4-byte instructions or 8 2-byte instructions, with bail-out circuitry to dispatch fewer instructions

by KMag 6y ago

How wasteful is it to have a bimodal decoder that assumes 8 4-byte instructions or 8 2-byte instructions, with bail-out circuitry to dispatch fewer instructions and advance to the next instruction-size boundary in the case of a stream of mixed widths? You don't need to handle the entire realm of possibilities in a single cycle if you're willing to pay a performance penalty for mixed-width instruction streams. You'd pay a performance penalty for mixed-width runs of instructions, but hopefully profile-guided-optimization would allow the compiler to know where to make the tradeoff for speed and were to make the tradeoff for size. Also, you mentioned elsewhere that you're a hardware designer. I'm aware that split register files are advantageous for speed, while unified integer and fp register files are advantageous for small/low power implementations. Given that the high-performance implementations are going to use register renaming anyway, has anyone quantified the penalty in using register renaming to allow for a split register file in high-performance implementations while allowing compatible low-power implementations with a single register file? Maybe the transistor savings are dwarfed by other concerns in a modern hardware implementation, but given RISC-V's academic goals, economy for FPGA implementation seems useful.

gchadwick 6y ago
> How wasteful is it to have a bimodal decoder that assumes 8 4-byte instructions or 8 2-byte instructions, with bail-out circuitry to dispatch fewer instructions and advance to the next instruction-size boundary in the case of a stream of mixed widths? I suspect mixed-width streams are extremely common (at least going by compiled RISC-V code I've seen). Potentially you can alter compilers to change this but the compressed instruction set is very limited, it naturally mixes compressed and uncompressed and forcing otherwise will likely produce poorer code overall. I'd also note that branch prediction accuracy is a key part of modern processor performance, they're maybe around 95-99% accurate most of the time. Your scheme looks to have similar mispredict penalties to branch prediction (you have to chuck things away and start over, though at least you don't have to refetch) so will likely require similar level of accuracy to work well. Plus on any mixed stream performance will be very poor as you've explicitly not built a decoder for mixed streams, so you end up dripping things through one at a time or you allow limited decoding on mixed streams (maybe 2 instructions per cycle). Edit: Perhaps the mis-predict penalty isn't so bad compared to a mis-predicted branch as detecting the mis-predict is trivial. Indeed having just realised this you may be better off without the predictor at all, just look at the bytes and decide which you should do. Though depending on the decoders you may end up needing two entirely separate sets of logic for the compressed instruction decoder vs uncompressed instruction decoder anyway, in which case you just decode your stream in both ways then decide which set of decodings you want to use. You still have the issue with any mixed width stream killing your performance though which I suspect would be a major issue with this design (so much so it's not useful).
jleahy 6y ago
> Has anyone quantified the penalty in using register renaming to allow for a split register file in high-performance implementations while allowing compatible low-power implementations with a single register file? Maybe the transistor savings are dwarfed by other concerns in a modern hardware implementation, but given RISC-V's academic goals, economy for FPGA implementation seems useful. I'm not sure I fully get your point. A low power implementation would have one physical register for each register in the instruction set (ie. for RISC-V it's already split). If you have a unified file in the instruction set then that means that you're going to have fewer GPRs available (ok for RISC-V as 16 is enough really) and you either need unified in your OOO implementation or you need to make integer->floatingpoint forwarding an exception (at which point you'd be better just to split 16/16).
- KMag 6y ago
  It's rare to have one function that uses both a lot of integer operations and a lot of fp operations. My understanding is that there are two advantages to having split gp and fp register files: (1) it is advantageous to not have integer/address calculations and fp operations not contend with each other over register file read ports (2) you get twice as many visible (architectural) registers without taking up more bits in the instruction stream. My understanding is that the designers of both POWER and aarch64 did a bunch of research before settling on 32 for the number of gp registers, and there are some workloads where 32 are really needed, so having a fixed split is suboptimal for some workloads. Aarch64 isn't that old, and I'm not aware of major improvements in compiler register allocation algorithms since it was designed. Are you contending that ARM made a mistake in their analysis when expanding from 16 to 31/32 gp registers for aarch64? So, my assumptions are (1) optimal split between fp and gp depends on workload (2) splitting them has a cost in die area for low-power implementations and (3) high performance chips can use the register renaming hardware to get the extra register file read ports from split register files and effectively tailor the gp/fp split to the workload. You wouldn't need to make integer-fp forwarding an exception, though the forwarding might have latency similar to an L1 cache read. Presumably, the register renaming hardware would keep a single bit (or small 2-3 bit saturating counter) to keep track of the last usage of each register, so that load instructions were likely to be stored to the optimal register file. It just seems that if you're going for a brand new green field architecture design, you can get rid of the cost of a split register file for low-power implementations and get all of the benefits of a split register file in high-performance designs that are going to have register renaming hardware anyway. Maybe I'm missing something with regard to your comment about integer-fp forwarding needing to be an exception instead of incurring slight latency.

by KMag 6y ago

gchadwick 6y ago

> How wasteful is it to have a bimodal decoder that assumes 8 4-byte instructions or 8 2-byte instructions, with bail-out circuitry to dispatch fewer instructions and advance to the next instruction-size boundary in the case of a stream of mixed widths? I suspect mixed-width streams are extremely common (at least going by compiled RISC-V code I've seen). Potentially you can alter compilers to change this but the compressed instruction set is very limited, it naturally mixes compressed and uncompressed and forcing otherwise will likely produce poorer code overall. I'd also note that branch prediction accuracy is a key part of modern processor performance, they're maybe around 95-99% accurate most of the time. Your scheme looks to have similar mispredict penalties to branch prediction (you have to chuck things away and start over, though at least you don't have to refetch) so will likely require similar level of accuracy to work well. Plus on any mixed stream performance will be very poor as you've explicitly not built a decoder for mixed streams, so you end up dripping things through one at a time or you allow limited decoding on mixed streams (maybe 2 instructions per cycle). Edit: Perhaps the mis-predict penalty isn't so bad compared to a mis-predicted branch as detecting the mis-predict is trivial. Indeed having just realised this you may be better off without the predictor at all, just look at the bytes and decide which you should do. Though depending on the decoders you may end up needing two entirely separate sets of logic for the compressed instruction decoder vs uncompressed instruction decoder anyway, in which case you just decode your stream in both ways then decide which set of decodings you want to use. You still have the issue with any mixed width stream killing your performance though which I suspect would be a major issue with this design (so much so it's not useful).

jleahy 6y ago

> Has anyone quantified the penalty in using register renaming to allow for a split register file in high-performance implementations while allowing compatible low-power implementations with a single register file? Maybe the transistor savings are dwarfed by other concerns in a modern hardware implementation, but given RISC-V's academic goals, economy for FPGA implementation seems useful. I'm not sure I fully get your point. A low power implementation would have one physical register for each register in the instruction set (ie. for RISC-V it's already split). If you have a unified file in the instruction set then that means that you're going to have fewer GPRs available (ok for RISC-V as 16 is enough really) and you either need unified in your OOO implementation or you need to make integer->floatingpoint forwarding an exception (at which point you'd be better just to split 16/16).

KMag 6y ago
It's rare to have one function that uses both a lot of integer operations and a lot of fp operations. My understanding is that there are two advantages to having split gp and fp register files: (1) it is advantageous to not have integer/address calculations and fp operations not contend with each other over register file read ports (2) you get twice as many visible (architectural) registers without taking up more bits in the instruction stream. My understanding is that the designers of both POWER and aarch64 did a bunch of research before settling on 32 for the number of gp registers, and there are some workloads where 32 are really needed, so having a fixed split is suboptimal for some workloads. Aarch64 isn't that old, and I'm not aware of major improvements in compiler register allocation algorithms since it was designed. Are you contending that ARM made a mistake in their analysis when expanding from 16 to 31/32 gp registers for aarch64? So, my assumptions are (1) optimal split between fp and gp depends on workload (2) splitting them has a cost in die area for low-power implementations and (3) high performance chips can use the register renaming hardware to get the extra register file read ports from split register files and effectively tailor the gp/fp split to the workload. You wouldn't need to make integer-fp forwarding an exception, though the forwarding might have latency similar to an L1 cache read. Presumably, the register renaming hardware would keep a single bit (or small 2-3 bit saturating counter) to keep track of the last usage of each register, so that load instructions were likely to be stored to the optimal register file. It just seems that if you're going for a brand new green field architecture design, you can get rid of the cost of a split register file for low-power implementations and get all of the benefits of a split register file in high-performance designs that are going to have register renaming hardware anyway. Maybe I'm missing something with regard to your comment about integer-fp forwarding needing to be an exception instead of incurring slight latency.