5 ms·

Do your tricks still apply to modern c++ compilers?

moonchild 4y ago
I cannot speak for c++, but I recently rewrote an optimised c routine in assembly. A 2-4x speedup obtained. I might have gotten partway with careful tuning of the source, the way the parent suggests, but I could have not gotten all the way.
favorited 4y ago
[Not GP] Modern optimizers are great, but writing cache-efficient code is still up to the programmer.
jandrewrogers 4y ago
Modern C++ compilers are mixed bag when it comes to optimization, brilliant in some areas and inexplicably obtuse in others requiring the programmer to be quite explicit. This has improved significantly with time but I am still sometimes surprised by the code the compiler has difficulty optimizing, or which are only recognized if written in very narrow ways. As a practical matter, compilers have a limited ability to reason over large areas of code in the way a programmer can but even very local optimizations are sometimes missed. This is why it is frequently helpful to look at the code generated by the C++ compiler. It gives you insight into the kinds of optimizations the compiler can see and which ones it can't, so you can focus on the ones it struggles with. This knowledge becomes out-of-date on the scale of years, so I periodically re-check what I think I know about what the compiler can optimize. For some things, like vectorization, the compiler's optimizer almost never produces a good result for non-trivial code and you'll have to do it yourself.
- orangepurple 4y ago
  I recommend using Godbolt (https://godbolt.org/ https://godbolt.org/) to view the assembly output of your compiled language (C++, etc)
  - zen_1 4y ago
    gdb's disas command (or asm view in general) is also very helpful, especially if you want to try control-c profiling [1] and then looking at the asm instructions in the neighborhood of where you halted your program. [1]: https://stackoverflow.com/questions/375913/how-can-i-profile-c-code-running-on-linux https://stackoverflow.com/questions/375913/how-can-i-profile...
  - ChrisMarshallNY 4y ago
    Godbolt is awesome. We used to use it all the time. The story I heard, is that the author threw it together to reinforce his side of an argument about code efficiency.
    - account42 4y ago
      > Godbolt is awesome. We used to use it all the time. I wonder how Matt Godbolt feels about being used or about being called "it". Joking of course - it was an odd choice to use $surname.org for one specific tool he built.
      - gpderetta 4y ago
        The tool is called Compiler Explorer; I think Matt just hosted it in his own otherwise unused existing personal domain, but the name famously stuck as it is short and memorable (at $WORK, our private compiler explorer installation is under go/godbolt!).
- jandrewrogers 4y ago
  I should add that some types of optimizations (cache efficiency being a big one) are outside the scope of the compiler because it is implicitly part of the code specification that the compiler needs to faithfully reproduce e.g. data structure layout is required to be a certain way for interoperability reasons.
  - moonchild 4y ago
    There was some work done a while ago on automatic layout optimisation. See chris lattner's dissertation. It seems to be mostly dead now, but the principles justifying its existence are not. Compilers can and do allow themselves license to ignore ABI in some cases; for c and c++, this includes link-time-optimised programs and -fwhole-program &c, as well as straight-up localised transformations on localised data structures (for whatever value of 'local' you prefer). Many other programming languages, in particular including JIT-compiled ones, also do not specify any ABI at all.
    - account42 4y ago
      For C++, the principle backing this is the as-if rule [0]. While compilers do optimize internal ABIs to some extend there is definitely still lots of room for improvement. I think eventually internal function boundaries and data structures should/will eventually only be hints to the optimizer just like the register and inline keywords have lost their original strict meaning - but we are not anywhere close to that yet. [0] https://en.cppreference.com/w/cpp/language/as_if https://en.cppreference.com/w/cpp/language/as_if
      - moonchild 4y ago
        When I say 'principles justifying its existence', I mean 'why we should do it', not 'why we may do it'.
- lumost 4y ago
  I’m curious how this compares in practice to other languages such as rust, Haskell, go, or Java. Cache efficiency at the level of arrays vs linked lists, and specific code for vectorization all happen in programmer land - but are C/C++ engineers really getting that much gain over alternative languages whose compilers don’t allow keywords for unlikely, or inline?
  - jandrewrogers 4y ago
    Attributes like inline or likely rarely do that much, so that doesn't explain it. They are merely suggestions, frequently ignored, and the compiler is good at optimizing these things anyway. Languages that allow explicit, deterministic, fine-grained resource management like C, C++, and Rust (systems languages) have significant advantages over ones that do not like Java and Go; certain types of optimizations cannot be effectively implemented without this property. In principle, systems languages can all produce similarly performant code but the amount of work required to get that performance varies greatly. Modern C++ in particular encourages the extensive use of expressive compile-time metaprogramming to automate optimization that the compiler cannot. In theory, the heavily optimized data structure and algorithm code that is often generated in C++ could be manually written in C or similar. However, in practice, few software engineers have the patience or time to systematically apply those optimizations manually to everything in their code base. I know for a fact that I didn't when I was writing C. That modern C++ tends to produce the most highly optimized code is more a matter of economy than theoretical capability.
    - lumost 4y ago
      Got it, I'll have to dig through the language benchmark game to see if it reflects the best C++ tricks. From what I've seen previously, the biggest perf difference is in native memory management along with easy access to machine specific instructions. Having recently worked on a large C/C++ code base which had a heavy focus on performance observation, I found myself wondering if tricks like dynamic Code Generation to avoid method overhead and unnecessary branch checks actually had a significant benefit on performance relative to the developer economy impact.
    - 4y ago
ChrisMarshallNY 4y ago
Probably, but I haven't done that stuff in over 5 years. We tended to use clang/llvm. I know that we often had to actively work against the native optimizations. We had a team from Intel, that used to run these awesome tools on our code, and helped us to see a lot of these weird dichotomies.

I cannot speak for c++, but I recently rewrote an optimised c routine in assembly. A 2-4x speedup obtained. I might have gotten partway with careful tuning of the source, the way the parent suggests, but I could have not gotten all the way.

favorited 4y ago

[Not GP] Modern optimizers are great, but writing cache-efficient code is still up to the programmer.

jandrewrogers 4y ago

Modern C++ compilers are mixed bag when it comes to optimization, brilliant in some areas and inexplicably obtuse in others requiring the programmer to be quite explicit. This has improved significantly with time but I am still sometimes surprised by the code the compiler has difficulty optimizing, or which are only recognized if written in very narrow ways. As a practical matter, compilers have a limited ability to reason over large areas of code in the way a programmer can but even very local optimizations are sometimes missed. This is why it is frequently helpful to look at the code generated by the C++ compiler. It gives you insight into the kinds of optimizations the compiler can see and which ones it can't, so you can focus on the ones it struggles with. This knowledge becomes out-of-date on the scale of years, so I periodically re-check what I think I know about what the compiler can optimize. For some things, like vectorization, the compiler's optimizer almost never produces a good result for non-trivial code and you'll have to do it yourself.

orangepurple 4y ago
I recommend using Godbolt (https://godbolt.org/ https://godbolt.org/) to view the assembly output of your compiled language (C++, etc)
- zen_1 4y ago
  gdb's disas command (or asm view in general) is also very helpful, especially if you want to try control-c profiling [1] and then looking at the asm instructions in the neighborhood of where you halted your program. [1]: https://stackoverflow.com/questions/375913/how-can-i-profile-c-code-running-on-linux https://stackoverflow.com/questions/375913/how-can-i-profile...
- ChrisMarshallNY 4y ago
  Godbolt is awesome. We used to use it all the time. The story I heard, is that the author threw it together to reinforce his side of an argument about code efficiency.
  - account42 4y ago
    > Godbolt is awesome. We used to use it all the time. I wonder how Matt Godbolt feels about being used or about being called "it". Joking of course - it was an odd choice to use $surname.org for one specific tool he built.
    - gpderetta 4y ago
      The tool is called Compiler Explorer; I think Matt just hosted it in his own otherwise unused existing personal domain, but the name famously stuck as it is short and memorable (at $WORK, our private compiler explorer installation is under go/godbolt!).
jandrewrogers 4y ago
I should add that some types of optimizations (cache efficiency being a big one) are outside the scope of the compiler because it is implicitly part of the code specification that the compiler needs to faithfully reproduce e.g. data structure layout is required to be a certain way for interoperability reasons.
- moonchild 4y ago
  There was some work done a while ago on automatic layout optimisation. See chris lattner's dissertation. It seems to be mostly dead now, but the principles justifying its existence are not. Compilers can and do allow themselves license to ignore ABI in some cases; for c and c++, this includes link-time-optimised programs and -fwhole-program &c, as well as straight-up localised transformations on localised data structures (for whatever value of 'local' you prefer). Many other programming languages, in particular including JIT-compiled ones, also do not specify any ABI at all.
  - account42 4y ago
    For C++, the principle backing this is the as-if rule [0]. While compilers do optimize internal ABIs to some extend there is definitely still lots of room for improvement. I think eventually internal function boundaries and data structures should/will eventually only be hints to the optimizer just like the register and inline keywords have lost their original strict meaning - but we are not anywhere close to that yet. [0] https://en.cppreference.com/w/cpp/language/as_if https://en.cppreference.com/w/cpp/language/as_if
    - moonchild 4y ago
      When I say 'principles justifying its existence', I mean 'why we should do it', not 'why we may do it'.
lumost 4y ago
I’m curious how this compares in practice to other languages such as rust, Haskell, go, or Java. Cache efficiency at the level of arrays vs linked lists, and specific code for vectorization all happen in programmer land - but are C/C++ engineers really getting that much gain over alternative languages whose compilers don’t allow keywords for unlikely, or inline?
- jandrewrogers 4y ago
  Attributes like inline or likely rarely do that much, so that doesn't explain it. They are merely suggestions, frequently ignored, and the compiler is good at optimizing these things anyway. Languages that allow explicit, deterministic, fine-grained resource management like C, C++, and Rust (systems languages) have significant advantages over ones that do not like Java and Go; certain types of optimizations cannot be effectively implemented without this property. In principle, systems languages can all produce similarly performant code but the amount of work required to get that performance varies greatly. Modern C++ in particular encourages the extensive use of expressive compile-time metaprogramming to automate optimization that the compiler cannot. In theory, the heavily optimized data structure and algorithm code that is often generated in C++ could be manually written in C or similar. However, in practice, few software engineers have the patience or time to systematically apply those optimizations manually to everything in their code base. I know for a fact that I didn't when I was writing C. That modern C++ tends to produce the most highly optimized code is more a matter of economy than theoretical capability.
  - lumost 4y ago
    Got it, I'll have to dig through the language benchmark game to see if it reflects the best C++ tricks. From what I've seen previously, the biggest perf difference is in native memory management along with easy access to machine specific instructions. Having recently worked on a large C/C++ code base which had a heavy focus on performance observation, I found myself wondering if tricks like dynamic Code Generation to avoid method overhead and unnecessary branch checks actually had a significant benefit on performance relative to the developer economy impact.
  - 4y ago

ChrisMarshallNY 4y ago

Probably, but I haven't done that stuff in over 5 years. We tended to use clang/llvm. I know that we often had to actively work against the native optimizations. We had a team from Intel, that used to run these awesome tools on our code, and helped us to see a lot of these weird dichotomies.