4 ms·

Ow. My head hurts. And this is why optimizing compilers are some of the most complex programs there are. (or so I have been taught)

by Malic 4y ago

shadowgovt 4y ago
It's also why the modern rule of thumb is "don't optimize by writing your own assembly." The rule is a boiled-down version of the larger notion "Don't optimize by writing your own assembly, because even with domain knowledge of the problem you're trying to solve, you're probably not more clever than the engineer-decades that went into building your compiler toolchain and processor architecture, unless you're an expert in both fields, in which case good luck and shoulder that maintenance burden." The rule of thumb drops a lot of detail on the ground but is a good first approximation.
- RodgerTheGreat 4y ago
  I would just phrase it as "if you optimize by writing your own assembly, don't expect your program or its performance to be portable."
- astrobe_ 4y ago
  ... Which is kind of worrying; is it really a good thing that processors are so complex that you need "decades" to use them to fully. Bottom line, you end up with chaotic (in the "sensitive to the slightest change") performance behavior. OTOH this reminds of another saying, "don't roll your own crypto". But all those "don't" are a bit frustrating.
  - miloignis 4y ago
    I've seen people get frustrated by the "don't"s before, but I think that's generally taking the first degree approximation too literally. Feel free to hand-write assembly or roll your own crypto, but don't depend on it for anything serious unless you are an expert or have it reviewed by one. Doing so for learning and fun is fine, if that's clearly called out such that no one accidentally depends on it for something serious. There's only one way to become good at something, and that's good practice! In a professional setting there's a responsibility to the end user which generally precludes doing these dangerous things - that is, one should feel free to take up woodworking as a hobby but shouldn't offer to build someone's house unless you're a licensed professional.
  - olliej 4y ago
    But you don't need decades of experience: we have compilers and optimizers to do that.
    - bioint7812 4y ago
      It's interesting that the impedence the author was experiencing was one of the CPU incorrectly "speculating" about the intent. We as readers are left to speculate about the problem being solved by the author. Based on the content of his recent articles, we could assume he is continuing his development of a JIT for a Rust port of his graphics engine. Given that assumption, I would argue that the compiler writers are lagging here--specifically lack of dynamic compilation. For example, is there a JIT compiler for the Rust language? I was thinking about reproducing his experiment in SBCL, which does have dynamic compilation--although it wouldn't be a real "apples" to "apples" comparison because my work machine is a x86_64.
      - olliej 4y ago
        JITs have very different constraints (but also many advantages) vs AOT compilers, so I don't think in the general case a language compiler can "support" JITs directly. llvm/clang for instance have a jit mode... for compiling C/C++, not some other language. Also obviously the compiler writers (AOT or JIT) need to know about and understand very minute details of how a cpu is behaving. I was responding to a person saying it was worrying that devs need that kind of knowledge and experience by saying that that isn't true because the tools exist (I feel that there's some analogy to what knowledge you need for a car: changing oil, servicing engine, replacing engine, designing an engine...). Once you are writing a JIT you're in the "making the tools" group, so you now need to know more than the vast majority of devs (not just more than the "average" dev) that's inescapable.
  - deleted 4y ago
    [deleted]
- pjdesno 4y ago
  Maybe better expressed as "don't write your own assembly unless you know why it might be better than the compiler". Trying to beat the compiler at optimizing normal code is kind of like trying to beat a calculator with paper and pencil - computers are just better at that sort of thing than people are. One use case is where you want to use bizarre CPU functions (popcount, encryption, load CR3, etc.) that no one's taught the compiler how to generate, although for some of them you might be better off using compiler intrinsics. Another is when you're dealing with things underneath the language abstraction, like the Boost co-routines mentioned in a link a few comments above. Of course, if the language folks decide to add the capability you want (e.g. C++20 coroutines), you're back to being better off using the compiler. Finally there are pedagogical reasons, e.g. sometimes I show my classes the world's simplest "Hello World", using a few lines of assembler to invoke the write and exit syscalls.
  - JonChesterfield 4y ago
    The boost coroutines mentioned are not the same thing as the C++20 ones. Boost captures the stack, that's what the register shuffling is for. C++ is a compiler transform that moves some state onto the heap, but probably not the whole stack, and builds a switch dispatch style thing to transform the control flow. This is why C++ comes with co_* annotations and coroutines don't.
- avgcorrection 4y ago
  I will never understand the trend of “using quotes around things”. Which is a shorthand version for “using quotes around things that I wanted to point to and say, hey, this is something that “goes over here”, you know, inside these quotes, to make sure that you understand exactly what I’m delimiting, since using commas, and semicolons, and colons wouldn’t fit for some reason. Oh look the thing that I’m quoting now consumes 80% of this paragraph. But this is way better than just saying “the modern rule of thumb is to not optimize by writing your own assembly.” Because then it isn’t 100% clear that the rule of thumb is delimited by (exactly) “[do] not optimize by writing your own assembly.” Ya see?”
- magicalhippo 4y ago
  Another is that your assembly will never target newer CPUs, but the compiler will. I've gained a lot of performance by replacing handwritten assembly functions with plain code versions, just because CPUs and compilers have evolved over the last 15-20 years, while that assembly code is what it is.
JonChesterfield 4y ago
There's a sense in which they're complicated. It's a sequence of graph transforms which mostly deal with non-polynomial time problems using heuristics, where mistakes in the transforms can manifest quite a long way away from the error. There's a significant risk that they're implemented in languages unique to that compiler toolchain, as compiler devs are quite prone to solving problems by writing compilers. There's also a sense in which they're really simple. The input format and output format are (usually) well defined. The user interface is largely printing things to stderr and giving up, possibly in a retry loop when there's an editor involved. The program dependency graph is often quite small so the bug you're looking at is probably in the source code you checked out. Security is not the dominant concern you have elsewhere.
bob1029 4y ago
The state of modern compilers is pretty staggering to me. I've seen some code folding in RyuJIT that makes me feel inferior as a developer. You've got a few compilers (Java, .NET, et. al.) which are capable of re-compiling hot path code during live execution and then seamlessly transitioning to those paths. This recompilation can be based upon the statistics of the live process, so it's almost like a sort of adaptive AI. Which paths are hot in production does not need to be known at compile time with these approaches.
astrange 4y ago
Optimizing compilers don't model things like branch prediction well, and aren't great at autovectorizing either. They work just well enough. In general I think they aren't that complicated since they have a pass structure that's relatively easy to inspect and helps avoid spaghetti code.

Ow. My head hurts. And this is why optimizing compilers are some of the most complex programs there are. (or so I have been taught)

by Malic 4y ago

Ow. My head hurts. And this is why optimizing compilers are some of the most complex programs there are. (or so I have been taught)

shadowgovt 4y ago

It's also why the modern rule of thumb is "don't optimize by writing your own assembly." The rule is a boiled-down version of the larger notion "Don't optimize by writing your own assembly, because even with domain knowledge of the problem you're trying to solve, you're probably not more clever than the engineer-decades that went into building your compiler toolchain and processor architecture, unless you're an expert in both fields, in which case good luck and shoulder that maintenance burden." The rule of thumb drops a lot of detail on the ground but is a good first approximation.

RodgerTheGreat 4y ago
I would just phrase it as "if you optimize by writing your own assembly, don't expect your program or its performance to be portable."
astrobe_ 4y ago
... Which is kind of worrying; is it really a good thing that processors are so complex that you need "decades" to use them to fully. Bottom line, you end up with chaotic (in the "sensitive to the slightest change") performance behavior. OTOH this reminds of another saying, "don't roll your own crypto". But all those "don't" are a bit frustrating.
- miloignis 4y ago
  I've seen people get frustrated by the "don't"s before, but I think that's generally taking the first degree approximation too literally. Feel free to hand-write assembly or roll your own crypto, but don't depend on it for anything serious unless you are an expert or have it reviewed by one. Doing so for learning and fun is fine, if that's clearly called out such that no one accidentally depends on it for something serious. There's only one way to become good at something, and that's good practice! In a professional setting there's a responsibility to the end user which generally precludes doing these dangerous things - that is, one should feel free to take up woodworking as a hobby but shouldn't offer to build someone's house unless you're a licensed professional.
- olliej 4y ago
  But you don't need decades of experience: we have compilers and optimizers to do that.
  - bioint7812 4y ago
    It's interesting that the impedence the author was experiencing was one of the CPU incorrectly "speculating" about the intent. We as readers are left to speculate about the problem being solved by the author. Based on the content of his recent articles, we could assume he is continuing his development of a JIT for a Rust port of his graphics engine. Given that assumption, I would argue that the compiler writers are lagging here--specifically lack of dynamic compilation. For example, is there a JIT compiler for the Rust language? I was thinking about reproducing his experiment in SBCL, which does have dynamic compilation--although it wouldn't be a real "apples" to "apples" comparison because my work machine is a x86_64.
    - olliej 4y ago
      JITs have very different constraints (but also many advantages) vs AOT compilers, so I don't think in the general case a language compiler can "support" JITs directly. llvm/clang for instance have a jit mode... for compiling C/C++, not some other language. Also obviously the compiler writers (AOT or JIT) need to know about and understand very minute details of how a cpu is behaving. I was responding to a person saying it was worrying that devs need that kind of knowledge and experience by saying that that isn't true because the tools exist (I feel that there's some analogy to what knowledge you need for a car: changing oil, servicing engine, replacing engine, designing an engine...). Once you are writing a JIT you're in the "making the tools" group, so you now need to know more than the vast majority of devs (not just more than the "average" dev) that's inescapable.
- deleted 4y ago
  [deleted]
pjdesno 4y ago
Maybe better expressed as "don't write your own assembly unless you know why it might be better than the compiler". Trying to beat the compiler at optimizing normal code is kind of like trying to beat a calculator with paper and pencil - computers are just better at that sort of thing than people are. One use case is where you want to use bizarre CPU functions (popcount, encryption, load CR3, etc.) that no one's taught the compiler how to generate, although for some of them you might be better off using compiler intrinsics. Another is when you're dealing with things underneath the language abstraction, like the Boost co-routines mentioned in a link a few comments above. Of course, if the language folks decide to add the capability you want (e.g. C++20 coroutines), you're back to being better off using the compiler. Finally there are pedagogical reasons, e.g. sometimes I show my classes the world's simplest "Hello World", using a few lines of assembler to invoke the write and exit syscalls.
- JonChesterfield 4y ago
  The boost coroutines mentioned are not the same thing as the C++20 ones. Boost captures the stack, that's what the register shuffling is for. C++ is a compiler transform that moves some state onto the heap, but probably not the whole stack, and builds a switch dispatch style thing to transform the control flow. This is why C++ comes with co_* annotations and coroutines don't.
avgcorrection 4y ago
I will never understand the trend of “using quotes around things”. Which is a shorthand version for “using quotes around things that I wanted to point to and say, hey, this is something that “goes over here”, you know, inside these quotes, to make sure that you understand exactly what I’m delimiting, since using commas, and semicolons, and colons wouldn’t fit for some reason. Oh look the thing that I’m quoting now consumes 80% of this paragraph. But this is way better than just saying “the modern rule of thumb is to not optimize by writing your own assembly.” Because then it isn’t 100% clear that the rule of thumb is delimited by (exactly) “[do] not optimize by writing your own assembly.” Ya see?”
magicalhippo 4y ago
Another is that your assembly will never target newer CPUs, but the compiler will. I've gained a lot of performance by replacing handwritten assembly functions with plain code versions, just because CPUs and compilers have evolved over the last 15-20 years, while that assembly code is what it is.

JonChesterfield 4y ago

There's a sense in which they're complicated. It's a sequence of graph transforms which mostly deal with non-polynomial time problems using heuristics, where mistakes in the transforms can manifest quite a long way away from the error. There's a significant risk that they're implemented in languages unique to that compiler toolchain, as compiler devs are quite prone to solving problems by writing compilers. There's also a sense in which they're really simple. The input format and output format are (usually) well defined. The user interface is largely printing things to stderr and giving up, possibly in a retry loop when there's an editor involved. The program dependency graph is often quite small so the bug you're looking at is probably in the source code you checked out. Security is not the dominant concern you have elsewhere.

bob1029 4y ago

The state of modern compilers is pretty staggering to me. I've seen some code folding in RyuJIT that makes me feel inferior as a developer. You've got a few compilers (Java, .NET, et. al.) which are capable of re-compiling hot path code during live execution and then seamlessly transitioning to those paths. This recompilation can be based upon the statistics of the live process, so it's almost like a sort of adaptive AI. Which paths are hot in production does not need to be known at compile time with these approaches.

astrange 4y ago

Optimizing compilers don't model things like branch prediction well, and aren't great at autovectorizing either. They work just well enough. In general I think they aren't that complicated since they have a pass structure that's relatively easy to inspect and helps avoid spaghetti code.