4 ms·

Oh man, that was fast. I didn't tell anybody I had released this yet, and here it is! Uhhh, AMA?

by _chris_ 10y ago

dbcurtis 10y ago
First, I am happy to see that you are attacking the correct problem. Everything interesting about computer architecture can be answered by asking two questions: 1) How is memory bandwidth used? (Seymour Cray lived and breathed that one every day), and 2) How is die area used? Achieving good code density is a double win -- less bandwidth for moving code into the I-cache, and reduced die area expended on I-cache to achieve a given amount of functionality. I'm curious how you would relate your work to WISC (Writable Instruction Store Computers) and how you view WISC machines?
- _chris_ 10y ago
  > I'm curious how you would relate your work to WISC (Writable Instruction Store Computers) and how you view WISC machines? You know, I've never heard of WISC machines before. I think the idea of RISC though is you are basically exposing the pipeline's micro-ops to the world, and the icache basically becomes your writeable "micro-code ROM", if you will. If you can't change how the pipeline's datapath is wired up, I'm not sure I'm clever enough to figure out how changing the instruction set would help (they needed writeable ucode in the old days, because they had so many bugs!). But with that said, one of my lab's goals has been to design and implement vector machine datapaths that are fairly configurable (say between program phases) in a way that can capture more types of compute kernels.
nkurz 10y ago
Thanks, I've skimmed though it and have a few questions and comments. I know very little about RISC-V, but I've recently been diving into issues of macro- and micro-fusion on recent Intel x64. The first question would be: Why do we care about instruction counts? Historically it's been used as a proxy for computational cost, but as you point out it no longer correlates that well. Additionally, many modern processors have µop caches that greatly reduce the importance of the initial instructions. But instead of suggesting that we ignore instruction count, the paper seems to suggest that we need to consider both instruction and µop count. Why? You mention that "Compiler tool chains are a continual work-in-progress" and mention that you "used GCC for all targets as it is widely used and the only compiler available for all systems". This makes me wonder how much you are testing the differences in ISA's versus how much you are just testing GCC's ability to optimize for each. My strong impression is that GCC does a poor job of optimizing macro- or micro-fusion for x64. It seems mostly oblivious to the issues, and the efficiency of the resulting code is equally likely to be anywhere from the same speed as microarchitecture conscious assembly to 2x slower. Did you attempt to correct for this when choosing x64 as the baseline? For example, I noticed the x64 code in 401.bzip in Code 2: n = ((Int32)block[ptr[unHi]+d]) - med; 4039d0: mov (%r10), %edx 4039d3: lea (%r15,%rdx,1), %eax 4039d7: movzbl (%r14,%rax,1), %eax 4039dc: sub %r9d, %eax 4039df: cmp $0x0, %eax 4039e2: jne 403a8a While some of this is due to the requirements of the specific integer widths (intentionally? arbitrarily? accidentally?) specified in the source, GCC seems to have come up with a very roundabout solution. Why a MOV/LEA rather than a micro-fused ADD load-op? For that matter, why use an LEA (which is restricted to fewer execution ports) rather than an ADD? And while in this case it happened to keep the CMP/JNE fusible, why does it have a CMP there at all, instead of using the flags already set by the SUB? From Sandy Bridge onward, SUB/JNE is also a macro-fused single instruction, so instead of two instructions fusing to one we have three instructions fusing to two. There are probably reasons for all this, but it does make it it awkward as a baseline. Whether it makes it to the paper or not, verifying that the baseline x64 numbers don't change significantly with Clang or ICC with a variety of optimization flags might be useful. If you were heroic, comparing to a version with hand-optimized would be interesting as well. A last comment is that I recently realized there were some significant differences between the Haswell and Skylake generations with regard to the effects of fusion on performance. A loop like this: #define ASM_MICRO_MACRO(in1, sum1, in2, sum2, max) \ __asm volatile ("1:\n" \ "add (%[IN1]), %[SUM1]\n" \ "cmp %[MAX], %[SUM1]\n" \ "jae 2f\n" \ "add (%[IN2]), %[SUM2]\n" \ "cmp %[MAX], %[SUM2]\n" \ "jb 1b\n" \ "2:" : \ [SUM1] "+&r" (sum1), \ [SUM2] "+&r" (sum2) : \ [IN1] "r" (in1), \ [IN2] "r" (in2), \ [MAX] "r" (max)) Ignoring that this isn't a particularly useful loop, it executes in at a single cycle per loop iteration on Skylake, but takes 1.8 cycles per iteration on Haswell. The amount of fusion is the same on each, but the performance differs greatly. I'm not yet sure yet of the specific reasons, but it makes me wonder if the choice of Ivy Bridge as the reference implementation for x64 might be influence your conclusions (although I don't know in which direction).
- mjevans 10y ago
  It sounds like Skylake is better at isolating sub-threads via result dependencies AND/OR it has more resources to utilize for your particular example. I wonder if it has some form of adaptation based on how likely it feels the parallelism is to pay off versus the power cost of that failure.
  - nkurz 10y ago
    I believe the difference is that Skylake removed a bottleneck on the number of µops that can be sustained at some stage other than the execution units, but I'm not sure where. In the example I gave, the ADD from memory is an example of micro-fusion. It's held as a single "complex" µop in the DSB, but is "un-laminated" in to 2 for execution and retirement. The CMP/JCC pairs use macro-fusion, and are combined to form a single µop both in the queue and during execution. It's documented that Skylake increased the number of µops that can be transferred from the DSB (Decoded Instruction Cache) to the IDQ (Instruction Decode Queue) from 4 µops per cycle to 6 µops per cycle. But by these metrics, we're only pulling 4 µops (representing 6 instructions) per cycle from the DSB, but these expand to 6 µops (not the same as the 6 instructions) at execution and retirement. I'm guessing that they improved the retirement bandwidth as well, but I haven't been able to find a clear statement of this, or what the new limits are.
- _chris_ 10y ago
  > But instead of suggesting that we ignore instruction count, the paper seems to suggest that we need to consider both instruction and µop count. Why? In part because I have nothing else to go on! Not every processor gives me a uop count. Instead I'm saying "let's look at instruction count, but with a glass full of salt." That may be fairly uncomforting, but it's what we have. But at the end of the day, I care about making fast RISC-V processors, and this level of analysis was enough to give me the insight I wanted that a) the gcc generation isn't too bad for RISC-V and b) hey, there's some neat tricks we can use to make fast RISC-V processors once we see what other ISAs thought was worth spending their opcode space on. > why does it have a CMP there at all, instead of using the flags already set by the SUB? I was quite surprised by how little x86 and ARMv7 made use of condition codes. Perhaps telling how ARMv8 removed most conditional execution from its ISA... > My strong impression is that GCC does a poor job of ... That may be (and I was certainly horrified by how badly it does on libquantum which is perfectly vectorizable). However, I've been told (for at least many SPEC benchmarks) that gcc does about as well as icc, and well, it's the free and available compiler that virtually everybody uses, so we're stuck with it! And the ISA has been around for 30+ years, so if it's still bad, is it the compiler's fault for not being smarter, or the ISA's fault for being too complex for the compiler to get it right? > If you were heroic, comparing to a version with hand-optimized would be interesting as well. There are a million things I'd love to have done (more benchmarks, more ISAs, more tool-chains...), but this isn't my PhD, and I had to draw the line somewhere! But if somebody wants a low-hanging Master's Thesis... > it makes me wonder if the choice of Ivy Bridge as the reference implementation for x64 might be influence your conclusions I'd love to read more about that. In my case, I use what I have on hand, which is a very expensive Xeon Ivy Bridge!

Oh man, that was fast. I didn't tell anybody I had released this yet, and here it is! Uhhh, AMA?

by _chris_ 10y ago

Oh man, that was fast. I didn't tell anybody I had released this yet, and here it is! Uhhh, AMA?

First, I am happy to see that you are attacking the correct problem. Everything interesting about computer architecture can be answered by asking two questions: 1) How is memory bandwidth used? (Seymour Cray lived and breathed that one every day), and 2) How is die area used? Achieving good code density is a double win -- less bandwidth for moving code into the I-cache, and reduced die area expended on I-cache to achieve a given amount of functionality. I'm curious how you would relate your work to WISC (Writable Instruction Store Computers) and how you view WISC machines?

_chris_ 10y ago
> I'm curious how you would relate your work to WISC (Writable Instruction Store Computers) and how you view WISC machines? You know, I've never heard of WISC machines before. I think the idea of RISC though is you are basically exposing the pipeline's micro-ops to the world, and the icache basically becomes your writeable "micro-code ROM", if you will. If you can't change how the pipeline's datapath is wired up, I'm not sure I'm clever enough to figure out how changing the instruction set would help (they needed writeable ucode in the old days, because they had so many bugs!). But with that said, one of my lab's goals has been to design and implement vector machine datapaths that are fairly configurable (say between program phases) in a way that can capture more types of compute kernels.

nkurz 10y ago

Thanks, I've skimmed though it and have a few questions and comments. I know very little about RISC-V, but I've recently been diving into issues of macro- and micro-fusion on recent Intel x64. The first question would be: Why do we care about instruction counts? Historically it's been used as a proxy for computational cost, but as you point out it no longer correlates that well. Additionally, many modern processors have µop caches that greatly reduce the importance of the initial instructions. But instead of suggesting that we ignore instruction count, the paper seems to suggest that we need to consider both instruction and µop count. Why? You mention that "Compiler tool chains are a continual work-in-progress" and mention that you "used GCC for all targets as it is widely used and the only compiler available for all systems". This makes me wonder how much you are testing the differences in ISA's versus how much you are just testing GCC's ability to optimize for each. My strong impression is that GCC does a poor job of optimizing macro- or micro-fusion for x64. It seems mostly oblivious to the issues, and the efficiency of the resulting code is equally likely to be anywhere from the same speed as microarchitecture conscious assembly to 2x slower. Did you attempt to correct for this when choosing x64 as the baseline? For example, I noticed the x64 code in 401.bzip in Code 2: n = ((Int32)block[ptr[unHi]+d]) - med; 4039d0: mov (%r10), %edx 4039d3: lea (%r15,%rdx,1), %eax 4039d7: movzbl (%r14,%rax,1), %eax 4039dc: sub %r9d, %eax 4039df: cmp $0x0, %eax 4039e2: jne 403a8a While some of this is due to the requirements of the specific integer widths (intentionally? arbitrarily? accidentally?) specified in the source, GCC seems to have come up with a very roundabout solution. Why a MOV/LEA rather than a micro-fused ADD load-op? For that matter, why use an LEA (which is restricted to fewer execution ports) rather than an ADD? And while in this case it happened to keep the CMP/JNE fusible, why does it have a CMP there at all, instead of using the flags already set by the SUB? From Sandy Bridge onward, SUB/JNE is also a macro-fused single instruction, so instead of two instructions fusing to one we have three instructions fusing to two. There are probably reasons for all this, but it does make it it awkward as a baseline. Whether it makes it to the paper or not, verifying that the baseline x64 numbers don't change significantly with Clang or ICC with a variety of optimization flags might be useful. If you were heroic, comparing to a version with hand-optimized would be interesting as well. A last comment is that I recently realized there were some significant differences between the Haswell and Skylake generations with regard to the effects of fusion on performance. A loop like this: #define ASM_MICRO_MACRO(in1, sum1, in2, sum2, max) \ __asm volatile ("1:\n" \ "add (%[IN1]), %[SUM1]\n" \ "cmp %[MAX], %[SUM1]\n" \ "jae 2f\n" \ "add (%[IN2]), %[SUM2]\n" \ "cmp %[MAX], %[SUM2]\n" \ "jb 1b\n" \ "2:" : \ [SUM1] "+&r" (sum1), \ [SUM2] "+&r" (sum2) : \ [IN1] "r" (in1), \ [IN2] "r" (in2), \ [MAX] "r" (max)) Ignoring that this isn't a particularly useful loop, it executes in at a single cycle per loop iteration on Skylake, but takes 1.8 cycles per iteration on Haswell. The amount of fusion is the same on each, but the performance differs greatly. I'm not yet sure yet of the specific reasons, but it makes me wonder if the choice of Ivy Bridge as the reference implementation for x64 might be influence your conclusions (although I don't know in which direction).

mjevans 10y ago
It sounds like Skylake is better at isolating sub-threads via result dependencies AND/OR it has more resources to utilize for your particular example. I wonder if it has some form of adaptation based on how likely it feels the parallelism is to pay off versus the power cost of that failure.
- nkurz 10y ago
  I believe the difference is that Skylake removed a bottleneck on the number of µops that can be sustained at some stage other than the execution units, but I'm not sure where. In the example I gave, the ADD from memory is an example of micro-fusion. It's held as a single "complex" µop in the DSB, but is "un-laminated" in to 2 for execution and retirement. The CMP/JCC pairs use macro-fusion, and are combined to form a single µop both in the queue and during execution. It's documented that Skylake increased the number of µops that can be transferred from the DSB (Decoded Instruction Cache) to the IDQ (Instruction Decode Queue) from 4 µops per cycle to 6 µops per cycle. But by these metrics, we're only pulling 4 µops (representing 6 instructions) per cycle from the DSB, but these expand to 6 µops (not the same as the 6 instructions) at execution and retirement. I'm guessing that they improved the retirement bandwidth as well, but I haven't been able to find a clear statement of this, or what the new limits are.
_chris_ 10y ago
> But instead of suggesting that we ignore instruction count, the paper seems to suggest that we need to consider both instruction and µop count. Why? In part because I have nothing else to go on! Not every processor gives me a uop count. Instead I'm saying "let's look at instruction count, but with a glass full of salt." That may be fairly uncomforting, but it's what we have. But at the end of the day, I care about making fast RISC-V processors, and this level of analysis was enough to give me the insight I wanted that a) the gcc generation isn't too bad for RISC-V and b) hey, there's some neat tricks we can use to make fast RISC-V processors once we see what other ISAs thought was worth spending their opcode space on. > why does it have a CMP there at all, instead of using the flags already set by the SUB? I was quite surprised by how little x86 and ARMv7 made use of condition codes. Perhaps telling how ARMv8 removed most conditional execution from its ISA... > My strong impression is that GCC does a poor job of ... That may be (and I was certainly horrified by how badly it does on libquantum which is perfectly vectorizable). However, I've been told (for at least many SPEC benchmarks) that gcc does about as well as icc, and well, it's the free and available compiler that virtually everybody uses, so we're stuck with it! And the ISA has been around for 30+ years, so if it's still bad, is it the compiler's fault for not being smarter, or the ISA's fault for being too complex for the compiler to get it right? > If you were heroic, comparing to a version with hand-optimized would be interesting as well. There are a million things I'd love to have done (more benchmarks, more ISAs, more tool-chains...), but this isn't my PhD, and I had to draw the line somewhere! But if somebody wants a low-hanging Master's Thesis... > it makes me wonder if the choice of Ivy Bridge as the reference implementation for x64 might be influence your conclusions I'd love to read more about that. In my case, I use what I have on hand, which is a very expensive Xeon Ivy Bridge!