6 ms·

C++ exception performance three years later

90 points by gpderetta 2y ago

fweimer 2y ago
For comparison, the original numbers are available in this C++ standardization paper: https://wg21.link/P2544 https://wg21.link/P2544
Mikhail_K 2y ago
TLDR: avoid using C++ exception if you can?
- gpderetta 2y ago
  On the contrary, after the scalability patches got through, the performance impact of exceptions even at high failure rate is in the noise.
- bayindirh 2y ago
  No, the only TLDR I got from that is, exceptions are should be reserved for exceptional cases. Their code is borderline abusing them, probably out of necessity. However, in this case this abuse provided a nice boost for everyone, so I can't complain.
- menaerus 2y ago
  As always, and generally speaking, if anything they show that drawing the line is difficult since they show in their paper [1] that both std::expected and boost::LEAF are worse than std::exception. Excerpts from paper: > Single threaded the fib code using std::expected is more than four times slower than using traditional exceptions. > This has much less overhead than std::expected, but it is still not for free. For fib we see a slowdown of approx. 60% compared to traditional exceptions, which is still problematic. Now, admittedly, these workloads are too simple (sqrt and fibonacci) and only shown for the single-threaded use-cases but I guess you have to start from something that is easy enough to reason about otherwise getting to any sane conclusions might be too difficult. However, they continue to show in their blog how they significantly sped up the std::exception implementation in a very non-trivial workload - multi-threaded and JIT'ed database kernel. So, to prove the point about exceptions being slower in complex workloads, such as the one above, somebody would actually have to rewrite their whole codebase to use something else such as std::expected, return values or whatever. Non-trivial to say the least. [1] https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2544r0.html https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p25...
- willvarfar 2y ago
  The article says: "Unwinding now scales with the number of cores, and we can safely use C++ exceptions even on machines with large core counts."
- protomolecule 2y ago
  Not at all. Basically, a naive implementation of exceptions uses a lock to guard unwind tables, and this scales poorly. The lock is needed only because .so libraries can be loaded and unloaded dynamically. One solution is to say that after a certain moment, the program doesn't load/unload dynamic libraries and doesn't use the lock after that. That's what ScyllaDB and Yandex[0] do, for example. Another solution is to just use the new glibc which fixes this issue. That aside, both exceptions and error codes introduce overhead, but they do it differently: exceptions add most[1] of their overhead when an error happens, while checking error codes adds overhead when an error doesn't happen[2]. One might conclude that exceptions should be used when errors are exceptionally rare; otherwise, use error codes. Or, just reclassify errors that happen too often as something you expect to happen, return them as a case of valid data and keep using exceptions. [0] https://habr.com/en/companies/yandex/articles/852244/ https://habr.com/en/companies/yandex/articles/852244/ [1] https://habr.com/en/companies/jugru/articles/494986/ https://habr.com/en/companies/jugru/articles/494986/ [2] https://archive.is/bIU23 https://archive.is/bIU23
  - adrian_b 2y ago
    Checking error codes adds overhead only in the way it is implemented in modern programming languages, by returning an integer error code that must be tested for a conditional jump after returning to the invoking procedure. Many early programming languages, including various dialects of Algol and Fortran, included a better implementation method. The invoking procedure passed to the invoked procedure not a single return address, but multiple return addresses. One return address was for the normal return and the other addresses were for alternate returns when errors were detected. On a modern CPU with many registers, all these return addresses would be passed in registers. The invoking procedure had at its end one or more handlers for error conditions, corresponding to the alternate return addresses. When a procedure was invoked, no tests were done at the invocation place, because any error would cause a jump to the error handler. In the invoked procedure, a test and a conditional jump must always exist in order to detect an error, and there the alternate return was executed only when an error was detected, so this implementation could omit one test and one conditional jump in comparison with the modern implementations. Moreover, the text of the procedure was more clear in this way, with all the error handlers separated from the normal execution case, but nonetheless close enough to examine them when necessary.
- bluGill 2y ago
  No use exceptions. In micro benchmarks we have known for years exceptions are slow. However more recient realistic benchmarks have discovered exceptions are generally faster than the checking - assuming you write all the if statements needed to unwind errors to where they can be handled - nobody does that in practice and so exceptions are slower than ignoring errors is what we were really saying when calling exceptions slow.
  - Leherenn 2y ago
    I have to admit I have a hard time relating your comment to my experience. I remember a case where we were using a library throwing exceptions when it could not parse some string. I think we were parsing IP addresses, or something like this. There was a case where this was called in a hot loop (like thousands of IP addresses in a roe) and for whatever reason in some cases most of them were invalid, thus throwing. Then simple error handling: the caller catches the exception, logs the error and checks the next one. The program was frozen for seconds. Replacing the parser with the same logic but returning a bool instead was something like 1000x faster. That was on MacOs Clang Arm. I don't see how a few extra ifs (though none were required in this case) would change anything. Generally I'm not a fan of exceptions, you never what something is going to throw. The majority of the crashes we see in production are unhandled exceptions. Turned out the documentation did not mention that this function could also throw std exceptions on top of the lib ones in this specific case.
    - saurik 2y ago
      You can always find a case where you are stressing the very thing that people are using in a micro-benchmark; like, no one is claiming that table unwinding is the correct choice for every single function! But, the other extreme you get is saying all error handling should be done with checks and branches, and the idea is that this slows down the code which isn't throwing lots of exceptions so much that, for average workloads, it is the wrong tradeoff. Ideally, you could make this decision more locally, and yet continue to use the cleaner syntax which comes from structured error handling, but I don't know of any language which implements this :/. And, so, the best we have is to just use a language which maps exception syntax to tables as you can always -- as you did -- fix the few places it matters to use a result type by adding manual checks.
- liontwist 2y ago
  Use C. The effective and productive language which is easier to understand than a single feature of C++ like exceptions or initialization.
- einpoklum 2y ago
  How about: "Make sure your performance-critical code / tight-loops don't do things which can throw exceptions, and then you won't have to worry (much) about the performance of exceptions" ? It may not be what the article says, but it's how I avoid having to delve into exception performance.
Dwedit 2y ago
Two different ideas of what an exception is. One idea is merely a form of control flow (like 'if' or 'goto') that could jump down the call stack. This kind of exception handler could be made fast if necessary. The other idea is to trigger the processor's actual fault handler, and proceed to the operating system, which then dispatches the exception to the program. The advantage of the latter is that you can catch exceptions raised by the processor that way. But since you need to enter the operating system's exception handler code, it goes slower. C# exceptions are like this.
- enjo 2y ago
  >One idea is merely a form of control flow (like 'if' or 'goto') that could jump down the call stack. Interestingly this is more or less how Symbian handled exceptions back in the day. It was a giant mess. You basically called setjmp in a surrounding function (using a macro IIRC) and then longjmp (in the form of user::leave()) which unwound the stack without calling any destructors. It was very fast and the source of endless memory leaks.
  - secondcoming 2y ago
    Not only were they fast but the codegen was very small. It had exception support before there was any real standardisation on how they should work. It was also before RAII was a thing and so you had to manage the ‘CleanupStack’ [0] yourself. At least for the development of the OS itself, and not frameworks such as the awful Series60 UI, we had have unit tests that brute forced memory correctness by getting the memory allocator to deliberately fail at each next step of the code being developed until it succeeded or panicked. [0] http://devlib.symbian.slions.net/s3/GUID-E7D29464-05E1-5039-8A02-62CB27B5FF21.html http://devlib.symbian.slions.net/s3/GUID-E7D29464-05E1-5039-...
jart 2y ago
It's worth mentioning this scalability issue doesn't apply to SJLJ exceptions. I wrote a program that throws an exception and catches it to warm up, and then throws a second exception. void first() { throw ""; } void second() { throw 2; } int main(int argc, char *argv[]) { try { first(); } catch (...) { } second(); } The second() function with LLVM libcxx using normal DWARF exceptions calls pthread_rwlock_rdlock SIX TIMES and calls pthread_rwlock_wrlock one time. Overall, this program triggers ~4000 function calls according to --ftrace (see https://justine.lol/ftrace/ https://justine.lol/ftrace/) On the other hand, if I use SJLJ, then this program triggers 269 function calls. It never locks anything. Although, like the first program, it does call pthread_once three times. That's doesn't cost anything though. It's the rwlocks you need to worry about. For an example of a modern C++ toolchain that uses SJLJ exceptions, see https://github.com/jart/cosmopolitan/releases/tag/3.7.1 https://github.com/jart/cosmopolitan/releases/tag/3.7.1 Please note that SJLJ exceptions carries a tradeoff. While SJLJ will make throwing exceptions much faster, you'll incur a slight performance degradation on normal code that isn't throwing exceptions. Using GCC in SJLJ mode will cause most of your C++ functions to have extra code inserted into the prologue and epilogue which calls _Unwind_SjLj_Register and _Unwind_SjLj_Unregister. Modern DWARF exception handling makes the intentional tradeoff of making exception throwing extremely slow, so that zero overhead is added for normal non-exceptional code. Both methods (SJLJ and DWARF) will bloat your binary size. With SJLJ it's just a few instructions inserted into prologues and epilogues. But with DWARF you get a lot of mandatory .eh_frame junk that's generated by everything, and it's centralized into a particular section of your binary that's only touched if exceptions are actually thrown.
- bryancoxwell 2y ago
  Can you expand on the concept of throwing/catching to warm up? I’ve never heard of that before (though I’m also not remotely familiar with C++).
  - achierius 2y ago
    Generally warming up caches (l1i, l1d, l2, ...) and other hardware state (e.g. out-of-order engine) so that you get a better picture of how your program will run in the steady state, rather than how it will perform on the first iteration with 'cold caches'. The justification for this is that in most (but not all) software applications, you should expect a reasonable amount of spatial/temporal locality and thus for your caches to be useful. So the 'average' performance of your program would be best captured by how the program runs when 'warm'. Warm -> populated with the code/data your program uses. Cold -> empty, or filled with stuff your program doesn't use. I do think that the case of exception handling is one where I would question the validity of this approach: if you catch exceptions rarely, it's quite reasonable to expect that the relevant lines would have been conflict'ed out of your l1i at some point in the interim, and therefore that the l1i (or even l2!) fetch cost is reasonable to include in the benchmark. However, if you expect your exceptions to fire 'relatively often' then it again becomes reasonable to warm up the caches before measuring, so it really comes down to the characteristics of your workload.
weinzierl 2y ago
From what I understand this about how the performance of code that throws scales with the number of cores. What is the state of current compilers for the non-throwing case? If I have code with exceptions, but test a case that does not actually throw? Will it be as fast as the same code but with exceptions removed altogether? EDIT: I just saw that Justine's (jart) comment answers this.
- adzm 2y ago
  Having exceptions as a possibility requires at least some bookkeeping and prevents certain kinds of optimizations due to the compiler not being able to know where an exception could come from. Generally however this is not that much of an issue, and only worth dealing with in the rarest of cases.
  - weinzierl 2y ago
    That makes sense. I wonder how the amount for this bookkeeping stacks up against the overhead of returning a sum type composed of error and result (like Rust Result<T, E>)? Always returning a slightly "fatter" type must certainly have an associated cost as well and it is to pay in any case - error or not.
    - gpderetta 2y ago
      At the limit, you can make Result<T,E> very cheap: with a specifically designed ABI you just need a bit in the flags register and a single (predictable) conditional jump on the return path (see for example the Herbceptions papers). At the limit limit, there is no difference between Result<T,E> and checked exceptions, with enough syntactic sugar they can be equivalent both semantically and syntactically [1]. [1] One advantage of typical C++ exceptions implementations, is that if the exception is not caught the stack is not unwound and the stack trace is preserved in the core file. In principle the same thing could be done for Result.
      - bluGill 2y ago
        With an exception (not checked exceptions) you get the advantage of the compiler knows about this and can optimize some code paths - any function that doesn't handle the function or have clean up to do can be bypassed, while result/checked exceptions need to go into that function anyway - which may be faster, more benchmarks needed. (though is possible a "sufficiently smart" compiler could understand thus pattern and turn it into an exception). So the real question is syntax - which do you prefer: having to write some result type all over instead of the regular return type you mean, or knowing that "magically" at unpredictable times your function will just abort in the middle. There are pros and cons to both, so anyone arguing absolutely for either side is wrong - at least now when we don't have decades of experience with result in large projects (though maybe we do in a single project)
e4m2 2y ago
Are there any benchmarks of Windows EH? The implementation is very different from DWARF/SJLJ EH and it would be interesting to see how it fares. I've seen some pretty exceptional claims for both sides of the argument (e.g. "[...] has no code impact for x64 native, ARM, or ARM64 platforms" [1]), but there's rarely if ever any data to back this up. [1] https://github.com/Microsoft/DirectXTK/wiki/throwIfFailed https://github.com/Microsoft/DirectXTK/wiki/throwIfFailed
- mananaysiempre 2y ago
  32-bit Windows SEH is very different for sure, but I thought 64-bit so-called “zero-overhead” SEH was a table-driven affair fairly similar to DWARF / Itanium ABI exceptions?
Someone 2y ago
Are the numbers in the second table correct? For 2, 4, 8 and 16 threads it shows a speedup for 0.1% failures vs 0% failures. For 2, 4, and 8 threads it also shows a speedup for 1% failures vs 0% failures. That speedup isn’t small. The largest one is for 8 threads, 0.1% vs 0%: 32 vs 64, so occasionally throwing exceptions seems to double the speed of that micro-benchmark. Is there so much noise in that data? If not, what could cause this?
- gpderetta 2y ago
  Noise from the jitter it seems.
lukaslalinsky 2y ago
I've been programming in Zig recently and I must say, that's the best error handling I've ever used in any programming language. They are just error return type, much like in Go or Rust, but the integration into the language makes it so seamless. It feels like using exceptions, but without the mental and computing overhead coming from the fact that anything can raise an exception.
- lbhdc 2y ago
  That has been my experience with zig too, it sparks a lot of joy.
- dataflow 2y ago
  > They are just error return type https://wg21.link/p2544#expected https://wg21.link/p2544#expected >> the overhead is so high that std::expected is not a good general purpose replacement for traditional exceptions. I'm guessing you don't care about performance that much.
  - feznyng 2y ago
    You might be right, but it doesn't seem correct to extrapolate that Zig's error return type is slow just because C++'s is. If Zig was designed with this kind of error handling from the start, it may well have optimized for it.
    - dataflow 2y ago
      > You might be right, but it doesn't seem correct to extrapolate that Zig's error return type is slow just because C++'s is. If Zig was designed with this kind of error handling from the start, it may well have optimized for it. It seems pretty correct to me? It's just based on first principles. Returning a value + error is strictly more work than returning just a value, and compilers can't solve the halting problem to optimize away arbitrary amounts of extra work. Ergo, performance hit.
      - MobiusHorizons 2y ago
dataflow 2y ago
From https://wg21.link/p2544 https://wg21.link/p2544 > The root cause is that the unwinder grabs a global mutex to protect the unwinding tables from concurrent changes from shared libraries. > We did a prototype implementation where we changed the gcc exception logic to register all unwinding tables in a b-tree with optimistic lock coupling. Did they try just stripping synchronization altogether to see what the performance ceiling would be?
- liontwist 2y ago
  Why not? That’s a good indicator of how much it’s costing, In the late 90s/early 2000s when languages started promoting general purpose threading, they did so without planning and simply threw locks on everything. Everyone now pays the cost of that feature whether you use it or not. Is throwing and catching exceptions across threads a good idea?
  - surajrmal 2y ago
    +1 there is so much unnecessary synchronization in things. It's crazy how much cost we pay for unnecessary locking in single threaded applications. Even multi threaded applications can end up scaling poorly. There are better ways to achieve partitioning of state that scales better than to simply litter granular locks everywhere.
PittleyDunkin 2y ago
Honest question: why use exceptions at all? They seem like far more hassle than benefit. Why not just use explicit error handling? This also results in much, much, much more readable code.
- kllrnohj 2y ago
  Because they are just better for the actual rare issue. Consider for example out of memory. "proper" error handling via return values would mean everything that allocates memory has to have an error return path and everything that calls anything that has that somewhere in the chain needs to still propagate that error. Essentially every function call now has a branch after it. If done explicitly, that's a lot of code nobody wants to write and, more importantly, that nobody ever actually tests. If done automatically, that's still just a metric shitload of branches which wastes branch predictor entries, it wastes cache size, etc... Exceptions, meanwhile, are typically free unless actually thrown. So for the truly 0.001% situation, like the aforementioned out of memory, they are perfect. You also get stack traces for free which greatly aid in debugging. Also typically you either handle errors pretty close to where the occur, or relatively high up at a more macro level. Exceptions lets your "middle" layer be significantly less branch-y & verbose for situations where it's a more macro-level error handling strategy. The alternative approach, like what Rust does, is to just say that rare errors like that are unrecoverable and hard-crash when encountered. That's certainly a choice you could make, but it's pretty heavy-handed for the language itself to mandate that. Which is then why Rust also provides std::catch_unwind, but with notes that it doesn't necessarily work because people can have chosen to compile with panic set to hard-abort period. It's pretty much a big mess.
  - PittleyDunkin 2y ago
    > "proper" error handling via return values would mean everything that allocates memory has to have an error return path and everything that calls anything that has that somewhere in the chain needs to still propagate that error. That seems like a win to me for the aforementioned "readable" criteria—failures should be enumerated. But I can see how that would be annoying to write. I'm also of the opinion that if handling out of memory is a serious consideration for your process, you should consider using your own allocator that doesn't fire an exception. This is how rust handles it, for instance. And on top of this I'm a little confused because exceptions are mostly disallowed from embedded codebases, which is the primary place where you would likely run out of memory and want to actually handle it.
pif 2y ago
In general, if you are benchmarking exceptions, it's a sign that you don't understand exceptions. Exceptions are not meant to replace error codes, that you use to get a detailed picture of the current environment. They are rather like a kind of assertions, where you don't actually care too much what happens next, because the exception being thrown means someone, somewhere, somehow, has already fuc*ed up.
- not_the_fda 2y ago
  Yep, exceptions are to be exceptional, i.e. rare.
- kllrnohj 2y ago
  Look at the error rates they are testing. A 0.1% exception rate, which is certainly something that'd be "rare", starts to happen an awful lot when you have a program that's using 128 cores, and 5th generation Epyc can push 384 threads in a single socket now.
  - pif 2y ago
    > Look at the error rates they are testing. I am sure they have very good reasons to keep working on this issue. That's why I started my comment with "in general".
- ryandrake 2y ago
  > Exceptions are not meant to replace error codes Just to state the obvious, this is a design philosophy (one of many) and not something imposed by the language. And reasonable people can disagree about whether it's a good one. I think where a lot of engineers go wrong is where they go crazy baking exception-throwing into their logic and error handling code, leaving a beautiful "happy path" in their code, but they are not disciplined about actually catching and handling the exceptions. If you're going to go all in on throwing, then you need to go all in on catching. Otherwise your product is just going to be full of unhandled crashing. A couple projects worth of this, and a developer will start to think "hmm, exceptions are bad, I agree we shouldn't use them!" Which is how we get blanket company policies against using any exceptions at all.

90 points by gpderetta 2y ago

For comparison, the original numbers are available in this C++ standardization paper: https://wg21.link/P2544 https://wg21.link/P2544

Mikhail_K 2y ago

TLDR: avoid using C++ exception if you can?

gpderetta 2y ago
On the contrary, after the scalability patches got through, the performance impact of exceptions even at high failure rate is in the noise.
bayindirh 2y ago
No, the only TLDR I got from that is, exceptions are should be reserved for exceptional cases. Their code is borderline abusing them, probably out of necessity. However, in this case this abuse provided a nice boost for everyone, so I can't complain.
menaerus 2y ago
As always, and generally speaking, if anything they show that drawing the line is difficult since they show in their paper [1] that both std::expected and boost::LEAF are worse than std::exception. Excerpts from paper: > Single threaded the fib code using std::expected is more than four times slower than using traditional exceptions. > This has much less overhead than std::expected, but it is still not for free. For fib we see a slowdown of approx. 60% compared to traditional exceptions, which is still problematic. Now, admittedly, these workloads are too simple (sqrt and fibonacci) and only shown for the single-threaded use-cases but I guess you have to start from something that is easy enough to reason about otherwise getting to any sane conclusions might be too difficult. However, they continue to show in their blog how they significantly sped up the std::exception implementation in a very non-trivial workload - multi-threaded and JIT'ed database kernel. So, to prove the point about exceptions being slower in complex workloads, such as the one above, somebody would actually have to rewrite their whole codebase to use something else such as std::expected, return values or whatever. Non-trivial to say the least. [1] https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2544r0.html https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p25...
willvarfar 2y ago
The article says: "Unwinding now scales with the number of cores, and we can safely use C++ exceptions even on machines with large core counts."
protomolecule 2y ago
Not at all. Basically, a naive implementation of exceptions uses a lock to guard unwind tables, and this scales poorly. The lock is needed only because .so libraries can be loaded and unloaded dynamically. One solution is to say that after a certain moment, the program doesn't load/unload dynamic libraries and doesn't use the lock after that. That's what ScyllaDB and Yandex[0] do, for example. Another solution is to just use the new glibc which fixes this issue. That aside, both exceptions and error codes introduce overhead, but they do it differently: exceptions add most[1] of their overhead when an error happens, while checking error codes adds overhead when an error doesn't happen[2]. One might conclude that exceptions should be used when errors are exceptionally rare; otherwise, use error codes. Or, just reclassify errors that happen too often as something you expect to happen, return them as a case of valid data and keep using exceptions. [0] https://habr.com/en/companies/yandex/articles/852244/ https://habr.com/en/companies/yandex/articles/852244/ [1] https://habr.com/en/companies/jugru/articles/494986/ https://habr.com/en/companies/jugru/articles/494986/ [2] https://archive.is/bIU23 https://archive.is/bIU23
- adrian_b 2y ago
  Checking error codes adds overhead only in the way it is implemented in modern programming languages, by returning an integer error code that must be tested for a conditional jump after returning to the invoking procedure. Many early programming languages, including various dialects of Algol and Fortran, included a better implementation method. The invoking procedure passed to the invoked procedure not a single return address, but multiple return addresses. One return address was for the normal return and the other addresses were for alternate returns when errors were detected. On a modern CPU with many registers, all these return addresses would be passed in registers. The invoking procedure had at its end one or more handlers for error conditions, corresponding to the alternate return addresses. When a procedure was invoked, no tests were done at the invocation place, because any error would cause a jump to the error handler. In the invoked procedure, a test and a conditional jump must always exist in order to detect an error, and there the alternate return was executed only when an error was detected, so this implementation could omit one test and one conditional jump in comparison with the modern implementations. Moreover, the text of the procedure was more clear in this way, with all the error handlers separated from the normal execution case, but nonetheless close enough to examine them when necessary.
bluGill 2y ago
No use exceptions. In micro benchmarks we have known for years exceptions are slow. However more recient realistic benchmarks have discovered exceptions are generally faster than the checking - assuming you write all the if statements needed to unwind errors to where they can be handled - nobody does that in practice and so exceptions are slower than ignoring errors is what we were really saying when calling exceptions slow.
- Leherenn 2y ago
  I have to admit I have a hard time relating your comment to my experience. I remember a case where we were using a library throwing exceptions when it could not parse some string. I think we were parsing IP addresses, or something like this. There was a case where this was called in a hot loop (like thousands of IP addresses in a roe) and for whatever reason in some cases most of them were invalid, thus throwing. Then simple error handling: the caller catches the exception, logs the error and checks the next one. The program was frozen for seconds. Replacing the parser with the same logic but returning a bool instead was something like 1000x faster. That was on MacOs Clang Arm. I don't see how a few extra ifs (though none were required in this case) would change anything. Generally I'm not a fan of exceptions, you never what something is going to throw. The majority of the crashes we see in production are unhandled exceptions. Turned out the documentation did not mention that this function could also throw std exceptions on top of the lib ones in this specific case.
  - saurik 2y ago
    You can always find a case where you are stressing the very thing that people are using in a micro-benchmark; like, no one is claiming that table unwinding is the correct choice for every single function! But, the other extreme you get is saying all error handling should be done with checks and branches, and the idea is that this slows down the code which isn't throwing lots of exceptions so much that, for average workloads, it is the wrong tradeoff. Ideally, you could make this decision more locally, and yet continue to use the cleaner syntax which comes from structured error handling, but I don't know of any language which implements this :/. And, so, the best we have is to just use a language which maps exception syntax to tables as you can always -- as you did -- fix the few places it matters to use a result type by adding manual checks.
liontwist 2y ago
Use C. The effective and productive language which is easier to understand than a single feature of C++ like exceptions or initialization.
einpoklum 2y ago
How about: "Make sure your performance-critical code / tight-loops don't do things which can throw exceptions, and then you won't have to worry (much) about the performance of exceptions" ? It may not be what the article says, but it's how I avoid having to delve into exception performance.

Dwedit 2y ago

Two different ideas of what an exception is. One idea is merely a form of control flow (like 'if' or 'goto') that could jump down the call stack. This kind of exception handler could be made fast if necessary. The other idea is to trigger the processor's actual fault handler, and proceed to the operating system, which then dispatches the exception to the program. The advantage of the latter is that you can catch exceptions raised by the processor that way. But since you need to enter the operating system's exception handler code, it goes slower. C# exceptions are like this.

enjo 2y ago
>One idea is merely a form of control flow (like 'if' or 'goto') that could jump down the call stack. Interestingly this is more or less how Symbian handled exceptions back in the day. It was a giant mess. You basically called setjmp in a surrounding function (using a macro IIRC) and then longjmp (in the form of user::leave()) which unwound the stack without calling any destructors. It was very fast and the source of endless memory leaks.
- secondcoming 2y ago
  Not only were they fast but the codegen was very small. It had exception support before there was any real standardisation on how they should work. It was also before RAII was a thing and so you had to manage the ‘CleanupStack’ [0] yourself. At least for the development of the OS itself, and not frameworks such as the awful Series60 UI, we had have unit tests that brute forced memory correctness by getting the memory allocator to deliberately fail at each next step of the code being developed until it succeeded or panicked. [0] http://devlib.symbian.slions.net/s3/GUID-E7D29464-05E1-5039-8A02-62CB27B5FF21.html http://devlib.symbian.slions.net/s3/GUID-E7D29464-05E1-5039-...

jart 2y ago

It's worth mentioning this scalability issue doesn't apply to SJLJ exceptions. I wrote a program that throws an exception and catches it to warm up, and then throws a second exception. void first() { throw ""; } void second() { throw 2; } int main(int argc, char *argv[]) { try { first(); } catch (...) { } second(); } The second() function with LLVM libcxx using normal DWARF exceptions calls pthread_rwlock_rdlock SIX TIMES and calls pthread_rwlock_wrlock one time. Overall, this program triggers ~4000 function calls according to --ftrace (see https://justine.lol/ftrace/ https://justine.lol/ftrace/) On the other hand, if I use SJLJ, then this program triggers 269 function calls. It never locks anything. Although, like the first program, it does call pthread_once three times. That's doesn't cost anything though. It's the rwlocks you need to worry about. For an example of a modern C++ toolchain that uses SJLJ exceptions, see https://github.com/jart/cosmopolitan/releases/tag/3.7.1 https://github.com/jart/cosmopolitan/releases/tag/3.7.1 Please note that SJLJ exceptions carries a tradeoff. While SJLJ will make throwing exceptions much faster, you'll incur a slight performance degradation on normal code that isn't throwing exceptions. Using GCC in SJLJ mode will cause most of your C++ functions to have extra code inserted into the prologue and epilogue which calls _Unwind_SjLj_Register and _Unwind_SjLj_Unregister. Modern DWARF exception handling makes the intentional tradeoff of making exception throwing extremely slow, so that zero overhead is added for normal non-exceptional code. Both methods (SJLJ and DWARF) will bloat your binary size. With SJLJ it's just a few instructions inserted into prologues and epilogues. But with DWARF you get a lot of mandatory .eh_frame junk that's generated by everything, and it's centralized into a particular section of your binary that's only touched if exceptions are actually thrown.

bryancoxwell 2y ago
Can you expand on the concept of throwing/catching to warm up? I’ve never heard of that before (though I’m also not remotely familiar with C++).
- achierius 2y ago
  Generally warming up caches (l1i, l1d, l2, ...) and other hardware state (e.g. out-of-order engine) so that you get a better picture of how your program will run in the steady state, rather than how it will perform on the first iteration with 'cold caches'. The justification for this is that in most (but not all) software applications, you should expect a reasonable amount of spatial/temporal locality and thus for your caches to be useful. So the 'average' performance of your program would be best captured by how the program runs when 'warm'. Warm -> populated with the code/data your program uses. Cold -> empty, or filled with stuff your program doesn't use. I do think that the case of exception handling is one where I would question the validity of this approach: if you catch exceptions rarely, it's quite reasonable to expect that the relevant lines would have been conflict'ed out of your l1i at some point in the interim, and therefore that the l1i (or even l2!) fetch cost is reasonable to include in the benchmark. However, if you expect your exceptions to fire 'relatively often' then it again becomes reasonable to warm up the caches before measuring, so it really comes down to the characteristics of your workload.

weinzierl 2y ago

From what I understand this about how the performance of code that throws scales with the number of cores. What is the state of current compilers for the non-throwing case? If I have code with exceptions, but test a case that does not actually throw? Will it be as fast as the same code but with exceptions removed altogether? EDIT: I just saw that Justine's (jart) comment answers this.

adzm 2y ago
Having exceptions as a possibility requires at least some bookkeeping and prevents certain kinds of optimizations due to the compiler not being able to know where an exception could come from. Generally however this is not that much of an issue, and only worth dealing with in the rarest of cases.
- weinzierl 2y ago
  That makes sense. I wonder how the amount for this bookkeeping stacks up against the overhead of returning a sum type composed of error and result (like Rust Result<T, E>)? Always returning a slightly "fatter" type must certainly have an associated cost as well and it is to pay in any case - error or not.
  - gpderetta 2y ago
    At the limit, you can make Result<T,E> very cheap: with a specifically designed ABI you just need a bit in the flags register and a single (predictable) conditional jump on the return path (see for example the Herbceptions papers). At the limit limit, there is no difference between Result<T,E> and checked exceptions, with enough syntactic sugar they can be equivalent both semantically and syntactically [1]. [1] One advantage of typical C++ exceptions implementations, is that if the exception is not caught the stack is not unwound and the stack trace is preserved in the core file. In principle the same thing could be done for Result.
    - bluGill 2y ago
      With an exception (not checked exceptions) you get the advantage of the compiler knows about this and can optimize some code paths - any function that doesn't handle the function or have clean up to do can be bypassed, while result/checked exceptions need to go into that function anyway - which may be faster, more benchmarks needed. (though is possible a "sufficiently smart" compiler could understand thus pattern and turn it into an exception). So the real question is syntax - which do you prefer: having to write some result type all over instead of the regular return type you mean, or knowing that "magically" at unpredictable times your function will just abort in the middle. There are pros and cons to both, so anyone arguing absolutely for either side is wrong - at least now when we don't have decades of experience with result in large projects (though maybe we do in a single project)

e4m2 2y ago

Are there any benchmarks of Windows EH? The implementation is very different from DWARF/SJLJ EH and it would be interesting to see how it fares. I've seen some pretty exceptional claims for both sides of the argument (e.g. "[...] has no code impact for x64 native, ARM, or ARM64 platforms" [1]), but there's rarely if ever any data to back this up. [1] https://github.com/Microsoft/DirectXTK/wiki/throwIfFailed https://github.com/Microsoft/DirectXTK/wiki/throwIfFailed

mananaysiempre 2y ago
32-bit Windows SEH is very different for sure, but I thought 64-bit so-called “zero-overhead” SEH was a table-driven affair fairly similar to DWARF / Itanium ABI exceptions?

Someone 2y ago

Are the numbers in the second table correct? For 2, 4, 8 and 16 threads it shows a speedup for 0.1% failures vs 0% failures. For 2, 4, and 8 threads it also shows a speedup for 1% failures vs 0% failures. That speedup isn’t small. The largest one is for 8 threads, 0.1% vs 0%: 32 vs 64, so occasionally throwing exceptions seems to double the speed of that micro-benchmark. Is there so much noise in that data? If not, what could cause this?

gpderetta 2y ago
Noise from the jitter it seems.

lukaslalinsky 2y ago

I've been programming in Zig recently and I must say, that's the best error handling I've ever used in any programming language. They are just error return type, much like in Go or Rust, but the integration into the language makes it so seamless. It feels like using exceptions, but without the mental and computing overhead coming from the fact that anything can raise an exception.

lbhdc 2y ago
That has been my experience with zig too, it sparks a lot of joy.
dataflow 2y ago
> They are just error return type https://wg21.link/p2544#expected https://wg21.link/p2544#expected >> the overhead is so high that std::expected is not a good general purpose replacement for traditional exceptions. I'm guessing you don't care about performance that much.
- feznyng 2y ago
  You might be right, but it doesn't seem correct to extrapolate that Zig's error return type is slow just because C++'s is. If Zig was designed with this kind of error handling from the start, it may well have optimized for it.
  - dataflow 2y ago
    > You might be right, but it doesn't seem correct to extrapolate that Zig's error return type is slow just because C++'s is. If Zig was designed with this kind of error handling from the start, it may well have optimized for it. It seems pretty correct to me? It's just based on first principles. Returning a value + error is strictly more work than returning just a value, and compilers can't solve the halting problem to optimize away arbitrary amounts of extra work. Ergo, performance hit.
    - MobiusHorizons 2y ago

dataflow 2y ago

From https://wg21.link/p2544 https://wg21.link/p2544 > The root cause is that the unwinder grabs a global mutex to protect the unwinding tables from concurrent changes from shared libraries. > We did a prototype implementation where we changed the gcc exception logic to register all unwinding tables in a b-tree with optimistic lock coupling. Did they try just stripping synchronization altogether to see what the performance ceiling would be?

liontwist 2y ago
Why not? That’s a good indicator of how much it’s costing, In the late 90s/early 2000s when languages started promoting general purpose threading, they did so without planning and simply threw locks on everything. Everyone now pays the cost of that feature whether you use it or not. Is throwing and catching exceptions across threads a good idea?
- surajrmal 2y ago
  +1 there is so much unnecessary synchronization in things. It's crazy how much cost we pay for unnecessary locking in single threaded applications. Even multi threaded applications can end up scaling poorly. There are better ways to achieve partitioning of state that scales better than to simply litter granular locks everywhere.

PittleyDunkin 2y ago

Honest question: why use exceptions at all? They seem like far more hassle than benefit. Why not just use explicit error handling? This also results in much, much, much more readable code.

kllrnohj 2y ago
Because they are just better for the actual rare issue. Consider for example out of memory. "proper" error handling via return values would mean everything that allocates memory has to have an error return path and everything that calls anything that has that somewhere in the chain needs to still propagate that error. Essentially every function call now has a branch after it. If done explicitly, that's a lot of code nobody wants to write and, more importantly, that nobody ever actually tests. If done automatically, that's still just a metric shitload of branches which wastes branch predictor entries, it wastes cache size, etc... Exceptions, meanwhile, are typically free unless actually thrown. So for the truly 0.001% situation, like the aforementioned out of memory, they are perfect. You also get stack traces for free which greatly aid in debugging. Also typically you either handle errors pretty close to where the occur, or relatively high up at a more macro level. Exceptions lets your "middle" layer be significantly less branch-y & verbose for situations where it's a more macro-level error handling strategy. The alternative approach, like what Rust does, is to just say that rare errors like that are unrecoverable and hard-crash when encountered. That's certainly a choice you could make, but it's pretty heavy-handed for the language itself to mandate that. Which is then why Rust also provides std::catch_unwind, but with notes that it doesn't necessarily work because people can have chosen to compile with panic set to hard-abort period. It's pretty much a big mess.
- PittleyDunkin 2y ago
  > "proper" error handling via return values would mean everything that allocates memory has to have an error return path and everything that calls anything that has that somewhere in the chain needs to still propagate that error. That seems like a win to me for the aforementioned "readable" criteria—failures should be enumerated. But I can see how that would be annoying to write. I'm also of the opinion that if handling out of memory is a serious consideration for your process, you should consider using your own allocator that doesn't fire an exception. This is how rust handles it, for instance. And on top of this I'm a little confused because exceptions are mostly disallowed from embedded codebases, which is the primary place where you would likely run out of memory and want to actually handle it.

pif 2y ago

In general, if you are benchmarking exceptions, it's a sign that you don't understand exceptions. Exceptions are not meant to replace error codes, that you use to get a detailed picture of the current environment. They are rather like a kind of assertions, where you don't actually care too much what happens next, because the exception being thrown means someone, somewhere, somehow, has already fuc*ed up.

not_the_fda 2y ago
Yep, exceptions are to be exceptional, i.e. rare.
kllrnohj 2y ago
Look at the error rates they are testing. A 0.1% exception rate, which is certainly something that'd be "rare", starts to happen an awful lot when you have a program that's using 128 cores, and 5th generation Epyc can push 384 threads in a single socket now.
- pif 2y ago
  > Look at the error rates they are testing. I am sure they have very good reasons to keep working on this issue. That's why I started my comment with "in general".
ryandrake 2y ago
> Exceptions are not meant to replace error codes Just to state the obvious, this is a design philosophy (one of many) and not something imposed by the language. And reasonable people can disagree about whether it's a good one. I think where a lot of engineers go wrong is where they go crazy baking exception-throwing into their logic and error handling code, leaving a beautiful "happy path" in their code, but they are not disciplined about actually catching and handling the exceptions. If you're going to go all in on throwing, then you need to go all in on catching. Otherwise your product is just going to be full of unhandled crashing. A couple projects worth of this, and a developer will start to think "hmm, exceptions are bad, I agree we shouldn't use them!" Which is how we get blanket company policies against using any exceptions at all.