5 ms·

How many die-hard assembly programmers today would do a better job in register allocation, instruction scheduling based on the latency of each individual instru

How many die-hard assembly programmers today would do a better job in register allocation, instruction scheduling based on the latency of each individual instruction, instruction selection to pick the instructions with the minimum encoded length, avoiding pipeline hazards or doing inlining decisions? These are hard optimization problems that require some real CPU juice to get right. An additional bonus is that the compiler does these really quickly for you and you don't have to worry about the instruction scheduling each time you refactor the code, for example.

dahart 10y ago
> How many die-hard assembly programmers today would do a better job... No idea. But I've done some assembly myself (as a console game programmer) and generally speaking, you don't have to think about any of the issues you raised to get faster code than what the compiler creates. There are some corner cases occasionally, but not very often. Does any of this mean I tend to choose assembly? No way. > the compiler does these really quickly for you It sounds like you generally agree with what I said? ;)
- JoeAltmaier 10y ago
  Depends upon the compiler. Might have been true in the past that compilers made worse code than a human. But today? I think compilers only get better at this. They would be hard to beat.
  - dahart 10y ago
    The console compilers (especially Nintendo's) do tend to be somewhat old & somewhat crappy. I haven't done game code in a couple of years, but for the decade that I did, the compilers on all platforms were 5-10 years behind current. That said, there are reasons that compilers often make slower code than humans, reasons that may always exist. The primary reason to use a compiler isn't to get faster code, it's to use a programming language abstraction and get "fast enough" code. The abstractions are (generally speaking) what add layers that are hard for a compiler to optimize against a person who's not using abstractions. There's a long list of commonly used features that compilers have a hard time optimizing: virtual functions, the threat of aliasing, floating point pipelining, etc. etc.. Compilers are required to make difference choices than humans writing assembly would, because of the abstractions that we choose to use in higher level languages. Lots of people agree that it's not hard to make assembly faster than an optimizing C compiler. http://stackoverflow.com/questions/577554/when-is-assembler-faster-than-c http://stackoverflow.com/questions/577554/when-is-assembler-...
    - JoeAltmaier 10y ago
      Yet, my very first C++ program was to declare an object that contained a byte, had a constructor to initialize that byte, and a method to return it. My 'main' declared an instance initialized with 'true' and the code simply returned the accessor method's value. The generated code? "mov ax, 255; ret" That was pretty good optimization, seeing completely through the abstraction and flattening it to the simplest possible implementation. I was impressed. Anyway there're lots of further abstractions these days, harder to see through. I imagine a clever human, with domain knowledge, can write a loop to do what a high-level library used to do.
      - dahart 10y ago
        This is a great example of something compilers definitely do better than humans writing assembly. Inlining code, especially with multiple passes, is something humans don't do. I still modularize in assembly and write functions, factor code used in multiple places into a single spot. The compiler is free to expand the same function inline a thousand places, and then use the expanded code to collapse to fewer instructions. The flattening can definitely be cool to see. So in your example, what would you have done if you'd written the assembly yourself? Do you feel like the compiler did something you wouldn't have?
        JoeAltmaier 10y ago
        Many programmers would have declared the object, set its members, and written main to fetch and return the data member. Not unreasonable, since humans tend to write abstractions pretty much as they're stated.
MichaelBurge 10y ago
There's not so much difference if you're writing C code, inspecting the resulting assembly, and tweaking the compiler flags and your data structures to get it to produce the assembly you want. But if you're not aware of what the compiler is emitting, it's often not hard to pull an order of magnitude improvement out of the code.
gens 10y ago
Funny thing is that one of the bigger problems with optimizing compilers is register allocation. Not only is it difficult to write a good algorithm for it, but compilers don't always know what variable is most important to keep in a register. Gcc even implemented a weaker allocator [0] as the older one was too complex to decipher problems with it. Instruction scheduling is not a problem on modern cpu's. Not that it's hard either way, just reorder things around memory access and heavy instructions like division. Agner Fog made a nice table of instruction latencies and throughputs [1]. Instruction encoding length... do you use 16bit integers or do you just int everything ? The compiler has to do what you tell it to do. Sure there are plenty of tricks around that but those are not the ones that give the compiler a remarkable advantage over humans. Tricks like using bitwise operations instead of multiplication and such and shortening equations is the advantage that the compiler has. But even that is not that big of a deal. (again, do you use the restrict keyword ? or static/define for constants ? do you know that C treats every "compile unit"(.c file) as a separate program ?) [0] https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=get&target=Local_Register_Allocator_Project_Detail.pdf https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=g... [1] http://www.agner.org/optimize/ http://www.agner.org/optimize/
- AstralStorm 10y ago
  I wonder if there is a resource for ARM CPUs like Agner's...
stephencanon 10y ago
I write assembly professionally, and I am significantly better at it than any compiler I've used. The considerations you list are all actually pretty easy to account for, but also mostly not very important for most code sequences on modern architectures(!) - Most performance-critical code is regular enough to stay far away from the NP-hard corners of register allocation. In the rare instances when I need to do careful hand-allocation to get an inner loop to fit perfectly, the compiler usually gives up and spills several variables. - Performance-critical code is rarely latency bound, and modern processors are wildly out-of-order, so scheduling based on latency is usually a waste of time (and sometimes a pessimization). Minimizing the critical path length of loop-carried dependencies and balancing port utilization are far more important factors in most code sequences. When code is latency-bound, it's often due to bad design decisions further up the stack; refactoring those to shift the concern to throughput is vastly more profitable than anything you can do to reduce latency locally, and is the sort of optimization that a compiler simply can't do (and usually can't even tell you you should do). I can tell the guy calling my BLAS library that he should try to batch up many vectors and call GEMM instead of GEMV. The compiler usually doesn't do that, and no-matter how hard it works at optimizing GEMV, it's going to be bound by memory access. - Encoding length is a factor for whole-program optimization (reducing i-cache pressure), but critical sections often fit into the decoded µop cache / loop buffer / inner cache / etc making this a non-issue. Sometimes you want to choose a particular encoding to optimize for e.g. fetch group alignment or to separate branches to aid the predictor, but there you're usually not trying to minimize the length but to get alignment without no-ops instead (and compilers generally don't do this optimization, because they don't have alignment/instruction size information until really late in the process). - Avoiding pipeline hazards and getting inlining right are are both worthwhile, but really, really straightforward for the most part. (Inlining is hard to get right if you're trying to avoid wasted work in the name of compiling as fast as you can, but humans don't have the restriction). The compiler works a lot faster than I do, but I'm a lot better at balancing these and other considerations and methodically optimizing a non-trivial code sequence until it's nearly optimal. I might take a week to optimize the same function that the compiler optimizes in a millisecond, but my code is faster. For most code, the compiler version is fine. For heavily-used system library functions, you definitely want my version instead. How can we improve compilers so that they're as good as I am? Compilers today mainly work in terms of local transforms on a computation graph, which necessarily cannot capture long-range optimizations that are available to me; most expert assembly programmers I know work mainly in terms of a global resource utilization minimization approach. There's no real reason why we can't make compilers that work that way, but it will almost surely make them significantly slower, so it would be more of a super-optimizer mode that you would only want to run once. I can also make inferences about what the programmer was "really" trying to do, and go down the hall to ask him. The compiler doesn't (usually) do that; there's a lot of opportunity for compilers to say "hey, I noticed that if x always has this common property, I can make your code much faster -- is that a valid assumption here?" Compilers aren't bad, but there's a lot of room for improvement, still.
dwringer 10y ago
I'm not sure it's an entirely apt comparison to put inefficient assembly programmers up against efficient C programmers. It may be much more difficult to get a skilled assembly programmer than it is to get a skilled C programmer, and the expectations of the assembly programmer may indeed be higher on a day-to-day basis, but saying C is faster than assembly because "Coder X" can write faster code in C doesn't really work for me.
pjc50 10y ago
I think you're going to be disappointed how few of those are handled for you by most compilers, unless you're using the Intel compiler and targeting a specific CPU.

dahart 10y ago

> How many die-hard assembly programmers today would do a better job... No idea. But I've done some assembly myself (as a console game programmer) and generally speaking, you don't have to think about any of the issues you raised to get faster code than what the compiler creates. There are some corner cases occasionally, but not very often. Does any of this mean I tend to choose assembly? No way. > the compiler does these really quickly for you It sounds like you generally agree with what I said? ;)

JoeAltmaier 10y ago
Depends upon the compiler. Might have been true in the past that compilers made worse code than a human. But today? I think compilers only get better at this. They would be hard to beat.
- dahart 10y ago
  The console compilers (especially Nintendo's) do tend to be somewhat old & somewhat crappy. I haven't done game code in a couple of years, but for the decade that I did, the compilers on all platforms were 5-10 years behind current. That said, there are reasons that compilers often make slower code than humans, reasons that may always exist. The primary reason to use a compiler isn't to get faster code, it's to use a programming language abstraction and get "fast enough" code. The abstractions are (generally speaking) what add layers that are hard for a compiler to optimize against a person who's not using abstractions. There's a long list of commonly used features that compilers have a hard time optimizing: virtual functions, the threat of aliasing, floating point pipelining, etc. etc.. Compilers are required to make difference choices than humans writing assembly would, because of the abstractions that we choose to use in higher level languages. Lots of people agree that it's not hard to make assembly faster than an optimizing C compiler. http://stackoverflow.com/questions/577554/when-is-assembler-faster-than-c http://stackoverflow.com/questions/577554/when-is-assembler-...
  - JoeAltmaier 10y ago
    Yet, my very first C++ program was to declare an object that contained a byte, had a constructor to initialize that byte, and a method to return it. My 'main' declared an instance initialized with 'true' and the code simply returned the accessor method's value. The generated code? "mov ax, 255; ret" That was pretty good optimization, seeing completely through the abstraction and flattening it to the simplest possible implementation. I was impressed. Anyway there're lots of further abstractions these days, harder to see through. I imagine a clever human, with domain knowledge, can write a loop to do what a high-level library used to do.
    - dahart 10y ago
      This is a great example of something compilers definitely do better than humans writing assembly. Inlining code, especially with multiple passes, is something humans don't do. I still modularize in assembly and write functions, factor code used in multiple places into a single spot. The compiler is free to expand the same function inline a thousand places, and then use the expanded code to collapse to fewer instructions. The flattening can definitely be cool to see. So in your example, what would you have done if you'd written the assembly yourself? Do you feel like the compiler did something you wouldn't have?
      - JoeAltmaier 10y ago
        Many programmers would have declared the object, set its members, and written main to fetch and return the data member. Not unreasonable, since humans tend to write abstractions pretty much as they're stated.

MichaelBurge 10y ago

There's not so much difference if you're writing C code, inspecting the resulting assembly, and tweaking the compiler flags and your data structures to get it to produce the assembly you want. But if you're not aware of what the compiler is emitting, it's often not hard to pull an order of magnitude improvement out of the code.

gens 10y ago

Funny thing is that one of the bigger problems with optimizing compilers is register allocation. Not only is it difficult to write a good algorithm for it, but compilers don't always know what variable is most important to keep in a register. Gcc even implemented a weaker allocator [0] as the older one was too complex to decipher problems with it. Instruction scheduling is not a problem on modern cpu's. Not that it's hard either way, just reorder things around memory access and heavy instructions like division. Agner Fog made a nice table of instruction latencies and throughputs [1]. Instruction encoding length... do you use 16bit integers or do you just int everything ? The compiler has to do what you tell it to do. Sure there are plenty of tricks around that but those are not the ones that give the compiler a remarkable advantage over humans. Tricks like using bitwise operations instead of multiplication and such and shortening equations is the advantage that the compiler has. But even that is not that big of a deal. (again, do you use the restrict keyword ? or static/define for constants ? do you know that C treats every "compile unit"(.c file) as a separate program ?) [0] https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=get&target=Local_Register_Allocator_Project_Detail.pdf https://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=g... [1] http://www.agner.org/optimize/ http://www.agner.org/optimize/

AstralStorm 10y ago
I wonder if there is a resource for ARM CPUs like Agner's...

stephencanon 10y ago

I write assembly professionally, and I am significantly better at it than any compiler I've used. The considerations you list are all actually pretty easy to account for, but also mostly not very important for most code sequences on modern architectures(!) - Most performance-critical code is regular enough to stay far away from the NP-hard corners of register allocation. In the rare instances when I need to do careful hand-allocation to get an inner loop to fit perfectly, the compiler usually gives up and spills several variables. - Performance-critical code is rarely latency bound, and modern processors are wildly out-of-order, so scheduling based on latency is usually a waste of time (and sometimes a pessimization). Minimizing the critical path length of loop-carried dependencies and balancing port utilization are far more important factors in most code sequences. When code is latency-bound, it's often due to bad design decisions further up the stack; refactoring those to shift the concern to throughput is vastly more profitable than anything you can do to reduce latency locally, and is the sort of optimization that a compiler simply can't do (and usually can't even tell you you should do). I can tell the guy calling my BLAS library that he should try to batch up many vectors and call GEMM instead of GEMV. The compiler usually doesn't do that, and no-matter how hard it works at optimizing GEMV, it's going to be bound by memory access. - Encoding length is a factor for whole-program optimization (reducing i-cache pressure), but critical sections often fit into the decoded µop cache / loop buffer / inner cache / etc making this a non-issue. Sometimes you want to choose a particular encoding to optimize for e.g. fetch group alignment or to separate branches to aid the predictor, but there you're usually not trying to minimize the length but to get alignment without no-ops instead (and compilers generally don't do this optimization, because they don't have alignment/instruction size information until really late in the process). - Avoiding pipeline hazards and getting inlining right are are both worthwhile, but really, really straightforward for the most part. (Inlining is hard to get right if you're trying to avoid wasted work in the name of compiling as fast as you can, but humans don't have the restriction). The compiler works a lot faster than I do, but I'm a lot better at balancing these and other considerations and methodically optimizing a non-trivial code sequence until it's nearly optimal. I might take a week to optimize the same function that the compiler optimizes in a millisecond, but my code is faster. For most code, the compiler version is fine. For heavily-used system library functions, you definitely want my version instead. How can we improve compilers so that they're as good as I am? Compilers today mainly work in terms of local transforms on a computation graph, which necessarily cannot capture long-range optimizations that are available to me; most expert assembly programmers I know work mainly in terms of a global resource utilization minimization approach. There's no real reason why we can't make compilers that work that way, but it will almost surely make them significantly slower, so it would be more of a super-optimizer mode that you would only want to run once. I can also make inferences about what the programmer was "really" trying to do, and go down the hall to ask him. The compiler doesn't (usually) do that; there's a lot of opportunity for compilers to say "hey, I noticed that if x always has this common property, I can make your code much faster -- is that a valid assumption here?" Compilers aren't bad, but there's a lot of room for improvement, still.

dwringer 10y ago

I'm not sure it's an entirely apt comparison to put inefficient assembly programmers up against efficient C programmers. It may be much more difficult to get a skilled assembly programmer than it is to get a skilled C programmer, and the expectations of the assembly programmer may indeed be higher on a day-to-day basis, but saying C is faster than assembly because "Coder X" can write faster code in C doesn't really work for me.

pjc50 10y ago

I think you're going to be disappointed how few of those are handled for you by most compilers, unless you're using the Intel compiler and targeting a specific CPU.