6 ms·

sigh If your application is slow, odds are it's not because you used exceptions. If you're throwing enough exceptions for this to matter it'll show up on a pro

by desc 7y ago

sigh If your application is slow, odds are it's not because you used exceptions. If you're throwing enough exceptions for this to matter it'll show up on a profiler, and then you can change that specific chunk of code to avoid treating that particular case as 'exceptional'.

abainbridge 7y ago
Yeah, it would be interesting to see the overhead exceptions add when no exception is thrown.
- abainbridge 7y ago
  Here's a godbolt link: https://godbolt.org/z/Z6vYAS https://godbolt.org/z/Z6vYAS Looking at the disassembly the machine code is ~2x the size for the exception versions, but most of it is on the cold path. The exception version has a conditional branch to do the "== errorInt" part. The non-exception version manages to avoid the conditional branch by using a conditional move, which would avoid a pipeline stall on a branch mis-prediction. Edit: I think this disproves desc's point ("If your application is slow <because of execptions> it'll show up on a profiler"). ie there's probably a small cost to exceptions even when they are not taken and it will be spread across your entire program and will not show as a single spike in a profiler.
  - abainbridge 7y ago
    The problem with benchmarks is that I never see any that estimate the impact of the extra code size on programs the size of, say, Photoshop. It takes annoying long to load such a program. Is code size part of that problem? Probably. Is the bloat added by exceptions significant? I'd like to know.
    - ncmncm 7y ago
      When it takes a program too long to load, it is because the program is doing too much non-exception work. The exception-handling code is not even being loaded unless it's throwing while it loads, which would just be bad design.
      - abainbridge 7y ago
        I think the exception code _is_ loaded. It is only a theoretical possibility that loading it could be avoided. I just built the following code with g++ v7.4 (from MSYS64 on Windows): #include <math.h> #include <stdexcept> void exitWithMessageException() { if (random() == 4321) throw std::runtime_error("Halt! Who goes there?"); } int main() { exitWithMessageException(); return 1234; } The generated code mixed the exception handlers with the hot-path code. Here are the address ranges of relevant chunks: 100401080 - Hot path of exitWithMessageException 100401099 10040109a - Cold path of exitWithMessageException 10040113f 100401140 - Start of main
        gpderetta 7y ago
        Interesting, GCC 7.x seems to simply puts the cold branch on a separate nop-padded cacheline. GCC 9 [1] instead moves the exception throwing branch into a cold clone of exitWithMessageException function. The behaviour seems to have changed on starting from GCC 8.x. [1] https://godbolt.org/z/PKKZ8m https://godbolt.org/z/PKKZ8m
        abainbridge 7y ago
        Ooo, fancy. There is still a long way from just that to actually getting the savings in a real program running on a real operating system. For example, if I have thousands of source files, each with a few hundred bytes of cold exception handlers, do they get coalesced into a single block for the whole program?
        gpderetta 7y ago
        coalescing functions in the same section should be the linker job, yes.
    - nurettin 7y ago
      Code paths introduced in order to execute any potential stack unrolling are inefficient and they make your code slow. Especially tight loops. This was common knowledge back in 2000s.
      - ncmncm 7y ago
        Common knowledge, but not correct. Code to destroy objects has to be generated for regular function returns, and is jumped into by the exception handler too. Managing resources by hand, instead, would also require code, but you have to write it. Its expense arises from its fragility.
        nurettin 7y ago
        What I meant was inserting stack frames into assembly, which are dissimilar to calling free, slowing things down.
  - gpderetta 7y ago
    > The non-exception version manages to avoid the conditional branch by using a conditional move, which would avoid a pipeline stall on a branch mis-prediction. branches are usually superior to conditional moves for predictable conditions as they break dependency chains. In case the exceptional code path is taken, the cost of the misprediciton is dwarfed by the cost of unwinding the stack. This is interesting actually, the fact that the compiler uses a conditional move in the error checking case could mean that the compiler has no useful branch probability model for that branch in the error checking case, but even when using __builtin_expect, the compiler still prefers the conditional move.
    - abainbridge 7y ago
      > branches are usually superior to conditional moves for predictable conditions as they break dependency chains. Interesting, not heard that before. Do you know of somewhere I can read about this?
      - gpderetta 7y ago
        Agner Fog is the usual go-to reference. For this specific case, you can also google any of Linus rants on conditional moves (they used to be very high latency, although today they are not so much of an issue). This one for example: https://yarchive.net/comp/linux/cmov.html https://yarchive.net/comp/linux/cmov.html
        abainbridge 7y ago
        Nice. Like every other topic, there's more complexity if you keep looking harder.
        ncmncm 7y ago
        It is complicated to describe when cmov is slow and when it is fast. As a rule of thumb, if the next loop iteration data operations depend on a cmov in this one, and around, cmov will be slow. If not, it is very, very fast. Use of cmov can make quicksort 2x as fast. Gcc absolutely won't generate two cmov instructions in a basic block. Clang, for its part, abandons practically all optimization of loops that could conceivably generate a throw.
  - lallysingh 7y ago
    But the relevant comparison is the cost of exception handling vs the cost of manual error checking. Of course if you don't check for or otherwise handle errors, the program will be faster. It's literally doing less work.
    - abainbridge 7y ago
      OK, another Godbolt link: https://godbolt.org/z/SiRvBR https://godbolt.org/z/SiRvBR This one adds functions that call the exception-based and error code based functions in a simple for loop. Both handle the error. Unless I've screwed up somewhere, I think the result is that in the exception case, the body of the inner loop contains 13 instructions, while the error code case contains 5. Also, the generated code for the exception case is harder to read and understand. When writing performance critical code I like to eye-ball the disassembly just to make sure the compiler didn't do anything unexpected. This task is hard enough already in non-trivial functions, I certainly don't want it getting any harder.
lovecg 7y ago
Some case might be “exceptional” in the sense that it doesn’t happen most of the time (and thus doesn’t show up on regular profile checks), but when it does happen the failures are highly correlated and suddenly you’re throwing thousands of exceptions per second all over the place. This is also the time one finds out that the exception handling code on typical C++ runtimes take out a global lock and the multithreaded application grinds to a halt—the raw CPU cost of an exception is not the most pressing problem at this point.
- AnimalMuppet 7y ago
  Wait, what? An exception takes out a global lock? Do you know which runtimes do this? Or, which runtimes don't?
  - gpderetta 7y ago
    libstc++ does. And IIRC the reason is the usual culprit: dlclose [1]. [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744
- BubRoss 7y ago
  malloc takes out a global lock and performance doesn't 'grind to a halt' from a single allocation. Hundreds of thousands of allocations per second per core and concurrency will suffer, but most programs either don't actually have nearly enough concurrency or minimize their allocations to the point where it isn't a primary bottleneck.
  - throwaway17_17 7y ago
    I think the context you’re working in matters heavily with such an assertion. In fact, it is quite common in game development to bulk allocate and then use custom allocators within these regions specifically avoid the drag on performance and determinism inflicted by malloc.
    - BubRoss 7y ago
munk-a 7y ago
Also generally having slow code is less a problem then expensive code - exceptions can allow you to use more succinct expression that will lower long term maintenance costs - the full lack of exceptions is one of the things that I think continues to impair Go which has a lot of other neat ideas (and a really good way to deal with sub-addressing in the form of slices).
- zozbot234 7y ago
  Doesn't Go have panic-recover which is basically the same deal?
  - munk-a 7y ago
    Panic/recover are much more limited and while you can build an exception system from them (much like all turing complete things can be all other turing complete things) it is a pain and extremely inefficient. So panics do exist and are used for I/O errors sometimes but it's quite inconsistent.
gameswithgo 7y ago
Tim Sweeney, creator of Unreal Engine, Epic Megagames, etc, recently had a code base where exceptions were causing problems even when not actively being thrown: https://twitter.com/TimSweeneyEpic/status/1223077404660371456?s=20 https://twitter.com/TimSweeneyEpic/status/122307740466037145... Quoting him in a follow up tweet: >They weren’t throwing and a disassembly showed no visible artifacts of exceptions. But turning off the possibility of throwing exceptions gained 15% and just made the assembly code tighter with no clear pattern to the improvements.
- desc 7y ago
  Indeed, and this was measured, and the improvement was worth it. Profiling! :P The cost isn't always where we think it is. Still, 'turn off exceptions because exceptions are slow' is a daft rule of thumb for the majority of software, where the slowness probably has more to do with choice of data structures, etc than compiler/platform implementation of language features. Always measure first, last, and in between.
- abainbridge 7y ago
  How does he know that the performance improvement wasn't just a side effect of altering the alignment of his code? Here's a guy getting a 48% performance improvement just by randomizing the alignment of his functions: https://youtu.be/Ho3bCIJcMcc?t=351 https://youtu.be/Ho3bCIJcMcc?t=351 or slides if you prefer https://github.com/dendibakh/dendibakh.github.io/blob/master/_posts/presentations/PerfAnalysisOnModernCPU.pdf https://github.com/dendibakh/dendibakh.github.io/blob/master...
  - gameswithgo 7y ago
    I am very familiar with this effect. We could of course use effects like this to cast doubt on any benchmark someone has ever run, unless they specifically mention that they tested for this, and 100 other bench marking gotchas. We can, if we like, assert that all things are unknowable, while at the same time asserting that the thing we want to be true is for sure true. However, it appears to be a rather commonplace occurrence that having exceptions on, even if you aren't using them, can cause performance problems, it isn't just a one of in Tim's case. Also Tim has quite a bit of experience working on bleeding edge C and C++ performance code so there is a good chance he did account for this. You can ask him.

by desc 7y ago

abainbridge 7y ago

Yeah, it would be interesting to see the overhead exceptions add when no exception is thrown.

abainbridge 7y ago
Here's a godbolt link: https://godbolt.org/z/Z6vYAS https://godbolt.org/z/Z6vYAS Looking at the disassembly the machine code is ~2x the size for the exception versions, but most of it is on the cold path. The exception version has a conditional branch to do the "== errorInt" part. The non-exception version manages to avoid the conditional branch by using a conditional move, which would avoid a pipeline stall on a branch mis-prediction. Edit: I think this disproves desc's point ("If your application is slow <because of execptions> it'll show up on a profiler"). ie there's probably a small cost to exceptions even when they are not taken and it will be spread across your entire program and will not show as a single spike in a profiler.
- abainbridge 7y ago
  The problem with benchmarks is that I never see any that estimate the impact of the extra code size on programs the size of, say, Photoshop. It takes annoying long to load such a program. Is code size part of that problem? Probably. Is the bloat added by exceptions significant? I'd like to know.
  - ncmncm 7y ago
    When it takes a program too long to load, it is because the program is doing too much non-exception work. The exception-handling code is not even being loaded unless it's throwing while it loads, which would just be bad design.
    - abainbridge 7y ago
      I think the exception code _is_ loaded. It is only a theoretical possibility that loading it could be avoided. I just built the following code with g++ v7.4 (from MSYS64 on Windows): #include <math.h> #include <stdexcept> void exitWithMessageException() { if (random() == 4321) throw std::runtime_error("Halt! Who goes there?"); } int main() { exitWithMessageException(); return 1234; } The generated code mixed the exception handlers with the hot-path code. Here are the address ranges of relevant chunks: 100401080 - Hot path of exitWithMessageException 100401099 10040109a - Cold path of exitWithMessageException 10040113f 100401140 - Start of main
      - gpderetta 7y ago
        Interesting, GCC 7.x seems to simply puts the cold branch on a separate nop-padded cacheline. GCC 9 [1] instead moves the exception throwing branch into a cold clone of exitWithMessageException function. The behaviour seems to have changed on starting from GCC 8.x. [1] https://godbolt.org/z/PKKZ8m https://godbolt.org/z/PKKZ8m
        abainbridge 7y ago
        Ooo, fancy. There is still a long way from just that to actually getting the savings in a real program running on a real operating system. For example, if I have thousands of source files, each with a few hundred bytes of cold exception handlers, do they get coalesced into a single block for the whole program?
        gpderetta 7y ago
        coalescing functions in the same section should be the linker job, yes.
  - nurettin 7y ago
    Code paths introduced in order to execute any potential stack unrolling are inefficient and they make your code slow. Especially tight loops. This was common knowledge back in 2000s.
    - ncmncm 7y ago
      Common knowledge, but not correct. Code to destroy objects has to be generated for regular function returns, and is jumped into by the exception handler too. Managing resources by hand, instead, would also require code, but you have to write it. Its expense arises from its fragility.
      - nurettin 7y ago
        What I meant was inserting stack frames into assembly, which are dissimilar to calling free, slowing things down.
- gpderetta 7y ago
  > The non-exception version manages to avoid the conditional branch by using a conditional move, which would avoid a pipeline stall on a branch mis-prediction. branches are usually superior to conditional moves for predictable conditions as they break dependency chains. In case the exceptional code path is taken, the cost of the misprediciton is dwarfed by the cost of unwinding the stack. This is interesting actually, the fact that the compiler uses a conditional move in the error checking case could mean that the compiler has no useful branch probability model for that branch in the error checking case, but even when using __builtin_expect, the compiler still prefers the conditional move.
  - abainbridge 7y ago
    > branches are usually superior to conditional moves for predictable conditions as they break dependency chains. Interesting, not heard that before. Do you know of somewhere I can read about this?
    - gpderetta 7y ago
      Agner Fog is the usual go-to reference. For this specific case, you can also google any of Linus rants on conditional moves (they used to be very high latency, although today they are not so much of an issue). This one for example: https://yarchive.net/comp/linux/cmov.html https://yarchive.net/comp/linux/cmov.html
      - abainbridge 7y ago
        Nice. Like every other topic, there's more complexity if you keep looking harder.
      - ncmncm 7y ago
        It is complicated to describe when cmov is slow and when it is fast. As a rule of thumb, if the next loop iteration data operations depend on a cmov in this one, and around, cmov will be slow. If not, it is very, very fast. Use of cmov can make quicksort 2x as fast. Gcc absolutely won't generate two cmov instructions in a basic block. Clang, for its part, abandons practically all optimization of loops that could conceivably generate a throw.
- lallysingh 7y ago
  But the relevant comparison is the cost of exception handling vs the cost of manual error checking. Of course if you don't check for or otherwise handle errors, the program will be faster. It's literally doing less work.
  - abainbridge 7y ago
    OK, another Godbolt link: https://godbolt.org/z/SiRvBR https://godbolt.org/z/SiRvBR This one adds functions that call the exception-based and error code based functions in a simple for loop. Both handle the error. Unless I've screwed up somewhere, I think the result is that in the exception case, the body of the inner loop contains 13 instructions, while the error code case contains 5. Also, the generated code for the exception case is harder to read and understand. When writing performance critical code I like to eye-ball the disassembly just to make sure the compiler didn't do anything unexpected. This task is hard enough already in non-trivial functions, I certainly don't want it getting any harder.

lovecg 7y ago

Some case might be “exceptional” in the sense that it doesn’t happen most of the time (and thus doesn’t show up on regular profile checks), but when it does happen the failures are highly correlated and suddenly you’re throwing thousands of exceptions per second all over the place. This is also the time one finds out that the exception handling code on typical C++ runtimes take out a global lock and the multithreaded application grinds to a halt—the raw CPU cost of an exception is not the most pressing problem at this point.

AnimalMuppet 7y ago
Wait, what? An exception takes out a global lock? Do you know which runtimes do this? Or, which runtimes don't?
- gpderetta 7y ago
  libstc++ does. And IIRC the reason is the usual culprit: dlclose [1]. [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744
BubRoss 7y ago
malloc takes out a global lock and performance doesn't 'grind to a halt' from a single allocation. Hundreds of thousands of allocations per second per core and concurrency will suffer, but most programs either don't actually have nearly enough concurrency or minimize their allocations to the point where it isn't a primary bottleneck.
- throwaway17_17 7y ago
  I think the context you’re working in matters heavily with such an assertion. In fact, it is quite common in game development to bulk allocate and then use custom allocators within these regions specifically avoid the drag on performance and determinism inflicted by malloc.
  - BubRoss 7y ago

munk-a 7y ago

Also generally having slow code is less a problem then expensive code - exceptions can allow you to use more succinct expression that will lower long term maintenance costs - the full lack of exceptions is one of the things that I think continues to impair Go which has a lot of other neat ideas (and a really good way to deal with sub-addressing in the form of slices).

zozbot234 7y ago
Doesn't Go have panic-recover which is basically the same deal?
- munk-a 7y ago
  Panic/recover are much more limited and while you can build an exception system from them (much like all turing complete things can be all other turing complete things) it is a pain and extremely inefficient. So panics do exist and are used for I/O errors sometimes but it's quite inconsistent.

gameswithgo 7y ago

Tim Sweeney, creator of Unreal Engine, Epic Megagames, etc, recently had a code base where exceptions were causing problems even when not actively being thrown: https://twitter.com/TimSweeneyEpic/status/1223077404660371456?s=20 https://twitter.com/TimSweeneyEpic/status/122307740466037145... Quoting him in a follow up tweet: >They weren’t throwing and a disassembly showed no visible artifacts of exceptions. But turning off the possibility of throwing exceptions gained 15% and just made the assembly code tighter with no clear pattern to the improvements.

desc 7y ago
Indeed, and this was measured, and the improvement was worth it. Profiling! :P The cost isn't always where we think it is. Still, 'turn off exceptions because exceptions are slow' is a daft rule of thumb for the majority of software, where the slowness probably has more to do with choice of data structures, etc than compiler/platform implementation of language features. Always measure first, last, and in between.
abainbridge 7y ago
How does he know that the performance improvement wasn't just a side effect of altering the alignment of his code? Here's a guy getting a 48% performance improvement just by randomizing the alignment of his functions: https://youtu.be/Ho3bCIJcMcc?t=351 https://youtu.be/Ho3bCIJcMcc?t=351 or slides if you prefer https://github.com/dendibakh/dendibakh.github.io/blob/master/_posts/presentations/PerfAnalysisOnModernCPU.pdf https://github.com/dendibakh/dendibakh.github.io/blob/master...
- gameswithgo 7y ago
  I am very familiar with this effect. We could of course use effects like this to cast doubt on any benchmark someone has ever run, unless they specifically mention that they tested for this, and 100 other bench marking gotchas. We can, if we like, assert that all things are unknowable, while at the same time asserting that the thing we want to be true is for sure true. However, it appears to be a rather commonplace occurrence that having exceptions on, even if you aren't using them, can cause performance problems, it isn't just a one of in Tim's case. Also Tim has quite a bit of experience working on bleeding edge C and C++ performance code so there is a good chance he did account for this. You can ask him.