4 ms·

The title is cutting out an important part of my report's story: "Avoiding ISA Bloat with Macro-Op Fusion". You can still specialize your processor to eke out

by _chris_ 10y ago

The title is cutting out an important part of my report's story: "Avoiding ISA Bloat with Macro-Op Fusion". You can still specialize your processor to eke out more performance, but you (often) don't need to change the ISA to do it! Why add load-pair instructions (like ARMv8 did) when you can simulate the exact same effects using macro-op fusion? And code size doesn't matter, since RISC-V's compressed ISA extension makes RISC-V the densest ISA out there without throwing away performance to get it.

ChuckMcM 10y ago
Let's start with I really like the RISC-V architecture, its compact, it is easy to understand and that it is "open" in the licensing sense is even more awesome. But I also think you've stepped into what I think of as the "known requirements" fallacy. Fundamentally the fallacy is this; When you are looking backwards in time at all of the accommodations existing systems made to adapt to changing requirements, you can see a path that those systems missed. It is similar perhaps to standing on the high ground over looking a valley and seeing a path that people on the ground do not see. The fallacy is believing that somehow you are a better pathfinder than they are and you would not make their choices in their position. Whether it was the saturating adds and multiplies that came with the DSP requirements, or the SIMD extensions that came with graphics, these systems live in an evolving ecosystem of computation which challenges their architectural invariants again and again. And commercially at least this pressure to adapt has so far had a perfect record of overwhelming the core architectural tenets and producing a small wart which later becomes a larger wart and then a system that, when you back on it, could have been built differently had they known the requirement that was going to be thrown at it. There will always be a case to throw out the currently dominant ISA and replace it with one without the warts of the existing system. And the cost of that will be high because it means a very large software migration burden. And it is that cost which allows the warts to exist in the first place. I think you have done some good work here. I particular like the analysis for the cost / benefit of adding instructions vs macro ops. I believe that will be a useful tool for doing architecture analysis going forward. I also found myself strongly rejecting the notion that that particular implementation of "macro op fusion" was sufficient to innoculate your ISA from any future changes. (which is implied by the abstract and endorsed by the conclusion)
- _chris_ 10y ago
  > The fallacy is believing that somehow you are a better pathfinder than they are and you would not make their choices in their position. RISC-V has two huge advantages going for it; 1) they got to learn from 40 years of mistakes, and 2) they recognized how important it was for the ISA to be agnostic to the micro-architecture (I'm amazed at how I can tell you about details such as the number and types of ports on a processor's register file just based on reading the ISA manual!). I don't think the RISC-V authors pretend they would not have made the same mistakes back then, but rather, they are annoyed that the same mistakes keep being made today! ARMv8 is a newer ISA than RISC-V and yet I believe it violates #2 (why are their branches 8 bytes?), and the same SIMD design mistakes are still being made over and over again. > I also found myself strongly rejecting the notion that that particular implementation of "macro op fusion" was sufficient to innoculate your ISA from any future changes. (which is implied by the abstract and endorsed by the conclusion) I don't mean to imply that there are no new instructions that will ever need to be added! I think popcount is an example that is very painful to handle in software, and the idiom in assembly is probably too big to handle via macro-op fusion. I'm also very excited to eventually have a vector ISA in RISC-V. Rather, my abstract is fighting against a whole slew of instructions that exist in current ISAs (or that people want to add to RISC-V) that don't need to be there. The load-pair/load-increment types that you see in ARMv8 come to mind.
  - ChuckMcM 10y ago
    Fair enough, have you asked the ARMv8 architect why he added them? He mentioned at ARM TechCon that there was a lot of thought that went into each change they made. [1] Richard Grisenthwaite, Lead Architect and Fellow. ARM -- https://www.arm.com/files/downloads/ARMv8_Architecture.pdf https://www.arm.com/files/downloads/ARMv8_Architecture.pdf
    - _chris_ 10y ago
      I would love to chat with the ARM guys and pick their brains. I suspect for a few things (like unfused branches) they wanted to match the existing micro-ops they were already generating since they would still have to support ARMv7! But I'd also like to know why they didn't have things like AMOs (and instead had to add them in v8.1). And lastly, I would like to know why there is no Thumb for AArch64, and why it isn't the default ARMv8 behavior!
cperciva 10y ago
Can you recognize and automatically perform macro-op fusion on AES computations? How about SHA256? Ok, let's make it easy -- CRC32?
Someone 10y ago
"And code size doesn't matter, since RISC-V's compressed ISA extension makes RISC-V the densest ISA out there without throwing away performance to get it." That doesn't rule out that there is some optimal compressed ISA that is a lot better than what RISC-V provides. As a truly weird idea, if a subroutine were tagged with the number of working registers it needs, instructions referencing registers inside such a subroutine could be shortened (ideally for memory usage using arithmetic coding, but shaving of integral bits probably would be easier on the decoder). Tagging need not take much room, as it could be applied to whole address ranges (MMU blocks, or even larger ones; for example, one could sacrifice the most significant bits of a 64-bit address range to it) if the linker or the loader moved subroutines with equal register usage together (but performance would suffer because of increased cache misses) Yes, that's a weird idea, and I would guess the gain in instruction density would not be enough to offset the extra cache misses and the added decoder complexity, and would not be particularly large to start with, but there may be real improvements. Maybe, one day, it will be efficient to gzip compress basic blocks or cache lines using a prefilled decoding table computed by the compiler?
- _chris_ 10y ago
  > That doesn't rule out that there is some optimal compressed ISA that is a lot better than what RISC-V provides. There's an entire PhD thesis on the RISC-V compressed ISA (RVC) and it shows that RVC's compression comes fairly close to just using bzip on the binary! > instructions referencing registers inside such a subroutine could be shortened They're already 3-bit register specifiers... there's not much more to do here. At this point, either your binary fits in the icache and you're happy or it doesn't and no amount of 1-5% ideas is going to help you.
  - Someone 10y ago
    "There's an entire PhD thesis on the RISC-V compressed ISA (RVC) and it shows that RVC's compression comes fairly close to just using bzip on the binary!" Thanks for the info, but I find it hard to believe that, so I would appreciate a link. Wouldn't that imply that we can create a compressor that is fairly close in quality to bzip and that is randomly addressable in O(1)?

by _chris_ 10y ago

ChuckMcM 10y ago

Let's start with I really like the RISC-V architecture, its compact, it is easy to understand and that it is "open" in the licensing sense is even more awesome. But I also think you've stepped into what I think of as the "known requirements" fallacy. Fundamentally the fallacy is this; When you are looking backwards in time at all of the accommodations existing systems made to adapt to changing requirements, you can see a path that those systems missed. It is similar perhaps to standing on the high ground over looking a valley and seeing a path that people on the ground do not see. The fallacy is believing that somehow you are a better pathfinder than they are and you would not make their choices in their position. Whether it was the saturating adds and multiplies that came with the DSP requirements, or the SIMD extensions that came with graphics, these systems live in an evolving ecosystem of computation which challenges their architectural invariants again and again. And commercially at least this pressure to adapt has so far had a perfect record of overwhelming the core architectural tenets and producing a small wart which later becomes a larger wart and then a system that, when you back on it, could have been built differently had they known the requirement that was going to be thrown at it. There will always be a case to throw out the currently dominant ISA and replace it with one without the warts of the existing system. And the cost of that will be high because it means a very large software migration burden. And it is that cost which allows the warts to exist in the first place. I think you have done some good work here. I particular like the analysis for the cost / benefit of adding instructions vs macro ops. I believe that will be a useful tool for doing architecture analysis going forward. I also found myself strongly rejecting the notion that that particular implementation of "macro op fusion" was sufficient to innoculate your ISA from any future changes. (which is implied by the abstract and endorsed by the conclusion)

_chris_ 10y ago
> The fallacy is believing that somehow you are a better pathfinder than they are and you would not make their choices in their position. RISC-V has two huge advantages going for it; 1) they got to learn from 40 years of mistakes, and 2) they recognized how important it was for the ISA to be agnostic to the micro-architecture (I'm amazed at how I can tell you about details such as the number and types of ports on a processor's register file just based on reading the ISA manual!). I don't think the RISC-V authors pretend they would not have made the same mistakes back then, but rather, they are annoyed that the same mistakes keep being made today! ARMv8 is a newer ISA than RISC-V and yet I believe it violates #2 (why are their branches 8 bytes?), and the same SIMD design mistakes are still being made over and over again. > I also found myself strongly rejecting the notion that that particular implementation of "macro op fusion" was sufficient to innoculate your ISA from any future changes. (which is implied by the abstract and endorsed by the conclusion) I don't mean to imply that there are no new instructions that will ever need to be added! I think popcount is an example that is very painful to handle in software, and the idiom in assembly is probably too big to handle via macro-op fusion. I'm also very excited to eventually have a vector ISA in RISC-V. Rather, my abstract is fighting against a whole slew of instructions that exist in current ISAs (or that people want to add to RISC-V) that don't need to be there. The load-pair/load-increment types that you see in ARMv8 come to mind.
- ChuckMcM 10y ago
  Fair enough, have you asked the ARMv8 architect why he added them? He mentioned at ARM TechCon that there was a lot of thought that went into each change they made. [1] Richard Grisenthwaite, Lead Architect and Fellow. ARM -- https://www.arm.com/files/downloads/ARMv8_Architecture.pdf https://www.arm.com/files/downloads/ARMv8_Architecture.pdf
  - _chris_ 10y ago
    I would love to chat with the ARM guys and pick their brains. I suspect for a few things (like unfused branches) they wanted to match the existing micro-ops they were already generating since they would still have to support ARMv7! But I'd also like to know why they didn't have things like AMOs (and instead had to add them in v8.1). And lastly, I would like to know why there is no Thumb for AArch64, and why it isn't the default ARMv8 behavior!

cperciva 10y ago

Can you recognize and automatically perform macro-op fusion on AES computations? How about SHA256? Ok, let's make it easy -- CRC32?

Someone 10y ago

"And code size doesn't matter, since RISC-V's compressed ISA extension makes RISC-V the densest ISA out there without throwing away performance to get it." That doesn't rule out that there is some optimal compressed ISA that is a lot better than what RISC-V provides. As a truly weird idea, if a subroutine were tagged with the number of working registers it needs, instructions referencing registers inside such a subroutine could be shortened (ideally for memory usage using arithmetic coding, but shaving of integral bits probably would be easier on the decoder). Tagging need not take much room, as it could be applied to whole address ranges (MMU blocks, or even larger ones; for example, one could sacrifice the most significant bits of a 64-bit address range to it) if the linker or the loader moved subroutines with equal register usage together (but performance would suffer because of increased cache misses) Yes, that's a weird idea, and I would guess the gain in instruction density would not be enough to offset the extra cache misses and the added decoder complexity, and would not be particularly large to start with, but there may be real improvements. Maybe, one day, it will be efficient to gzip compress basic blocks or cache lines using a prefilled decoding table computed by the compiler?

_chris_ 10y ago
> That doesn't rule out that there is some optimal compressed ISA that is a lot better than what RISC-V provides. There's an entire PhD thesis on the RISC-V compressed ISA (RVC) and it shows that RVC's compression comes fairly close to just using bzip on the binary! > instructions referencing registers inside such a subroutine could be shortened They're already 3-bit register specifiers... there's not much more to do here. At this point, either your binary fits in the icache and you're happy or it doesn't and no amount of 1-5% ideas is going to help you.
- Someone 10y ago
  "There's an entire PhD thesis on the RISC-V compressed ISA (RVC) and it shows that RVC's compression comes fairly close to just using bzip on the binary!" Thanks for the info, but I find it hard to believe that, so I would appreciate a link. Wouldn't that imply that we can create a compressor that is fairly close in quality to bzip and that is randomly addressable in O(1)?