30 ms·
Investigating the Performance Overhead of C++ Exceptions
- jimbob45 7y agoTL;DR: We rediscovered monads.
- gpderetta 7y agoWell yes. And that's why there's a group that has in the committee that has been trying to steer await towards a proper do-notation.
- desc 7y agosigh If your application is slow, odds are it's not because you used exceptions. If you're throwing enough exceptions for this to matter it'll show up on a profiler, and then you can change that specific chunk of code to avoid treating that particular case as 'exceptional'.
- abainbridge 7y agoYeah, it would be interesting to see the overhead exceptions add when no exception is thrown.
- abainbridge 7y agoHere's a godbolt link: https://godbolt.org/z/Z6vYAS https://godbolt.org/z/Z6vYAS Looking at the disassembly the machine code is ~2x the size for the exception versions, but most of it is on the cold path. The exception version has a conditional branch to do the "== errorInt" part. The non-exception version manages to avoid the conditional branch by using a conditional move, which would avoid a pipeline stall on a branch mis-prediction. Edit: I think this disproves desc's point ("If your application is slow <because of execptions> it'll show up on a profiler"). ie there's probably a small cost to exceptions even when they are not taken and it will be spread across your entire program and will not show as a single spike in a profiler.
- abainbridge 7y agoThe problem with benchmarks is that I never see any that estimate the impact of the extra code size on programs the size of, say, Photoshop. It takes annoying long to load such a program. Is code size part of that problem? Probably. Is the bloat added by exceptions significant? I'd like to know.
- ncmncm 7y agoWhen it takes a program too long to load, it is because the program is doing too much non-exception work. The exception-handling code is not even being loaded unless it's throwing while it loads, which would just be bad design.
- abainbridge 7y agoI think the exception code _is_ loaded. It is only a theoretical possibility that loading it could be avoided. I just built the following code with g++ v7.4 (from MSYS64 on Windows): #include <math.h> #include <stdexcept> void exitWithMessageException() { if (random() == 4321) throw std::runtime_error("Halt! Who goes there?"); } int main() { exitWithMessageException(); return 1234; } The generated code mixed the exception handlers with the hot-path code. Here are the address ranges of relevant chunks: 100401080 - Hot path of exitWithMessageException 100401099 10040109a - Cold path of exitWithMessageException 10040113f 100401140 - Start of main
- gpderetta 7y agoInteresting, GCC 7.x seems to simply puts the cold branch on a separate nop-padded cacheline. GCC 9 [1] instead moves the exception throwing branch into a cold clone of exitWithMessageException function. The behaviour seems to have changed on starting from GCC 8.x. [1] https://godbolt.org/z/PKKZ8m https://godbolt.org/z/PKKZ8m
- abainbridge 7y ago
- lovecg 7y agoSome case might be “exceptional” in the sense that it doesn’t happen most of the time (and thus doesn’t show up on regular profile checks), but when it does happen the failures are highly correlated and suddenly you’re throwing thousands of exceptions per second all over the place. This is also the time one finds out that the exception handling code on typical C++ runtimes take out a global lock and the multithreaded application grinds to a halt—the raw CPU cost of an exception is not the most pressing problem at this point.
- AnimalMuppet 7y agoWait, what? An exception takes out a global lock? Do you know which runtimes do this? Or, which runtimes don't?
- gpderetta 7y agolibstc++ does. And IIRC the reason is the usual culprit: dlclose [1]. [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744
- BubRoss 7y agomalloc takes out a global lock and performance doesn't 'grind to a halt' from a single allocation. Hundreds of thousands of allocations per second per core and concurrency will suffer, but most programs either don't actually have nearly enough concurrency or minimize their allocations to the point where it isn't a primary bottleneck.
- throwaway17_17 7y agoI think the context you’re working in matters heavily with such an assertion. In fact, it is quite common in game development to bulk allocate and then use custom allocators within these regions specifically avoid the drag on performance and determinism inflicted by malloc.
- BubRoss 7y ago
- munk-a 7y agoAlso generally having slow code is less a problem then expensive code - exceptions can allow you to use more succinct expression that will lower long term maintenance costs - the full lack of exceptions is one of the things that I think continues to impair Go which has a lot of other neat ideas (and a really good way to deal with sub-addressing in the form of slices).
- zozbot234 7y agoDoesn't Go have panic-recover which is basically the same deal?
- munk-a 7y agoPanic/recover are much more limited and while you can build an exception system from them (much like all turing complete things can be all other turing complete things) it is a pain and extremely inefficient. So panics do exist and are used for I/O errors sometimes but it's quite inconsistent.
- gameswithgo 7y agoTim Sweeney, creator of Unreal Engine, Epic Megagames, etc, recently had a code base where exceptions were causing problems even when not actively being thrown: https://twitter.com/TimSweeneyEpic/status/1223077404660371456?s=20 https://twitter.com/TimSweeneyEpic/status/122307740466037145... Quoting him in a follow up tweet: >They weren’t throwing and a disassembly showed no visible artifacts of exceptions. But turning off the possibility of throwing exceptions gained 15% and just made the assembly code tighter with no clear pattern to the improvements.
- desc 7y agoIndeed, and this was measured, and the improvement was worth it. Profiling! :P The cost isn't always where we think it is. Still, 'turn off exceptions because exceptions are slow' is a daft rule of thumb for the majority of software, where the slowness probably has more to do with choice of data structures, etc than compiler/platform implementation of language features. Always measure first, last, and in between.
- abainbridge 7y agoHow does he know that the performance improvement wasn't just a side effect of altering the alignment of his code? Here's a guy getting a 48% performance improvement just by randomizing the alignment of his functions: https://youtu.be/Ho3bCIJcMcc?t=351 https://youtu.be/Ho3bCIJcMcc?t=351 or slides if you prefer https://github.com/dendibakh/dendibakh.github.io/blob/master/_posts/presentations/PerfAnalysisOnModernCPU.pdf https://github.com/dendibakh/dendibakh.github.io/blob/master...
- gameswithgo 7y agoI am very familiar with this effect. We could of course use effects like this to cast doubt on any benchmark someone has ever run, unless they specifically mention that they tested for this, and 100 other bench marking gotchas. We can, if we like, assert that all things are unknowable, while at the same time asserting that the thing we want to be true is for sure true. However, it appears to be a rather commonplace occurrence that having exceptions on, even if you aren't using them, can cause performance problems, it isn't just a one of in Tim's case. Also Tim has quite a bit of experience working on bleeding edge C and C++ performance code so there is a good chance he did account for this. You can ask him.
- CodeMage 7y agoI was disappointed they didn't do a test case with std::error_code. EDIT: Is there something wrong with std::error_code? I don't care about my points, but I'm worried I might have missed something I should know.
- ncmncm 7y agoIt would reveal nothing meaningful. We know how slow handling status codes is, and we know it is equally slow, pass or fail.
- Thorrez 7y agoI personally have a hard time understanding std::error_code. Specifically I can't figure out when std::generic_category vs std::system_category should be used, and I worry that confusion between the 2 of them will lead to confusing and inconsistent APIs where it's hard to predict what error_code will be returned in what situation.
- vlovich123 7y agostd::system_category and std::generic_category are generally the same on non-Windows platforms. std::system_category on Windows is any Windows error (HRESULT, GetLastError, etc) and std::generic_category always represents errno. So for cross-platform APIs that exist on both you'll generally use std::generic_category as the API is usually defined to modify errno in the case of an error. For Windows-only APIs use std::system_category.
- krapht 7y agoIf you wanted this today, why would you implement the Expected template instead of using Boost Outcome?
- detaro 7y agoFor the blogpost it makes more sense showing an example implementation IMHO. And in some places, it's easier just writing it out than deal with the process to add a dependency.
- thedance 7y agoIt should be well-known that actually throwing an exception in C++ is extremely costly. I think it is also fairly widespread knowledge that even allowing the compiler to support exceptions has runtime costs. Exceptions do not conform to C++'s "zero-overhead" principle. That's why numerous large organizations from NASA to the Dept. of Defense to Google decree that C++ exceptions must not be used.
- dana321 7y agoIts unrolling the stack, whatever could go wrong?.. Its only expensive when you throw an exception. They should only be used in exceptional circumstances, but they appear to be littered among libraries for everything from telling the caller that the end of file has occurred to a connection timing out.
- thedance 7y agoThe cost comes from having to track what automatic and temporary objects have been created and attaching them to the exception context so you can destroy them if an exception is thrown. Another cost is having to track which base classes of a derived class have already been constructed when the derived constructor throws. You get noticeably better generated code when the compiler can skip all that junk.
- AnimalMuppet 7y agoWait a minute. > The cost comes from having to track what automatic and temporary objects have been created and attaching them to the exception context so you can destroy them if an exception is thrown. Doesn't it have to do that anyway, in case you return from a function, or break from inside a loop? Anytime you can leave a scope, it has to be able to do that. > Another cost is having to track which base classes of a derived class have already been constructed when the derived constructor throws. All of them have been constructed before you begin the derived constructor.
- thedance 7y ago
- arunc 7y agoExceptions are expensive and I hope no body uses it in the hot path (80%). The compiler has to inject a lot of code to unwind the stack that affects the cache. It is however not expensive for common operations (20%). There are couple of threads in dforum that discusses this and to have throw attribute [1] and eventually making nothrow as default [2]. [1] https://forum.dlang.org/thread/sbdrybtyfkxfhxxjgqco@forum.dlang.org https://forum.dlang.org/thread/sbdrybtyfkxfhxxjgqco@forum.dl... [2] https://forum.dlang.org/post/qur0l0$2h8s$1@digitalmars.com https://forum.dlang.org/post/qur0l0$2h8s$1@digitalmars.com
- lallysingh 7y ago(quoting desc) sigh (end quote) Exceptions make your code faster in the non-exceptional case. See all those if-branches for error checks that don't exist in your code? That branch-free code is going to run faster than your branchy code. And the code's shorter, because it doesn't have the checks and handling intermixed, which makes it fit better into cache. This isn't rocket science, I'm amazed that this isn't obvious.
- abainbridge 7y agoCheck out the godbolt link I posted under desc's thread. The answer isn't that obvious. Both approaches have a conditional. The compiler/linker marks the exception handling code as cold. In theory this could go in a section of the exe that the loader never pages into RAM. (I'm getting out of my depth here).
- thedance 7y agoYeah because that's going to go well: taking a major page fault whenever you throw. Let me know how that works at scale.
- cwzwarich 7y agoYou already take a page fault with DWARF exceptions, since the unwind / LSDA info is probably on a different page from any of your code, as well as the personality routine itself.
- lallysingh 7y agoIf you're throwing, performance isn't the concern. You just want to clean up, return your 500 or what not, and get back to a sane state.
- gpderetta 7y agothat potentially happen even with normal if based error handing. If the compiler thinks the error handling path is unlikely to be run, it can move it to a cold page.
- dyingkneepad 7y agoI would love to see the results for increasingly values of randomRange. This is just benchmarking the case when you throw exceptions 50% of the time.
- ratboy666 7y agoConverting an exception into a return. Which loses the "have to deal with it; cannot ignore it" property that made the exception valuable. "The key takeaway here is that an exception cannot be ignored or forgotten; if we have an exception, it must be dealt with." If converted to simple "C style" status returns -- it is only 5 times slower (11.5ns vs 57.5ns) -- and can be ignored and forgotten. In the associated slide deck, boolean valid() must be used before value. In this case, this is just as unsafe as C return codes. A C++ exception is always considered an lvalue, meaning that union is initialized, but I am not sure what reading the other side results in (I don't think that this is allowed). My take on this? Please use an exception if you mean an exception (for C++). If it is too slow (which I consider unlikely), please deal with it some other way (I would suggest a status return).
- patrec 7y ago> Converting an exception into a return. Which loses the "have to deal with it; cannot ignore it" property that made the exception valuable. [[nodiscard]] -Werror
- jcelerier 7y ago> "The key takeaway here is that an exception cannot be ignored or forgotten; if we have an exception, it must be dealt with." but a good way to deal with exceptions (in the very specific case of GUI desktop software, though that works also for server software that handles requests) is to just have a global catch handler at the top of your event loop which logs an error message / displays a message box to your user / does a preemptive backup save of the data in case something is deeply corrupted somewhere and then restarts the event loop.
- fefe23 7y agoExceptions are not meant to be fast. In fact, exceptions are meant to be exceptional. Hence the name. Exceptions are meant to be fast if and only if you do not throw any. And even this is only truly true on Unix. The original implementation of exceptions on Windows put extra unrolling info on the stack and then cleaned it up at the end of the function, adding a runtime cost even when no exceptions were thrown. The Unix way of exception handling does not put anything extra on the stack, so exceptions are free if you don't throw any. The compiler puts some metadata into your binary and generates cleanup code. If an exception is thrown, the run-time (libstdc++ for gcc on Linux) will grab the return address from the stack and then do a binary search through the metadata to find the right unwinding code generated by the compiler. This is obviously a comparatively enormous overhead. In particular because the metadata are separated from the code, so the OS never has to read it from disk unless you actually throw an exception. The reason for that is that you are not supposed to use exceptions for signaling. They are for error handling. Exceptions are for occasions like "you asked for memory but there was none left" or "you asked for the 15th element in this vector but it only has 10". They are not meant for non-exceptional errors like "you asked for data from the network but there is no data yet" or "you wanted me to check if the file exists and it does not". The thinking behind all this is that the error handling path is not time critical. If it happens, something exceptional has happened, and the OS will probably have to swap in the error handling code from disk to begin with, which is horrendously slow. In fact, this is not just a possibility. The feature was designed specifically so that the linker would move the exception handling code to a different page and the OS would only read that page from disk if an exception actually occurs. This is not an accident. It is deliberate planning and took engineering effort to achieve. Running a benchmark of throw vs return is an attempt to answer the wrong question. Important side note: operator[] on a vector does not do range checking. at() does. That's why you should always use at() instead of operator[].
- deleted 7y ago[deleted]
- quietbritishjim 7y agoYou seem to be arguing that, because exception tables end up on separate pages, we should consider exceptions as being as expensive as disk accesses. (Apologies if I've just built a straw man, that's how I read it.) I don't buy that. If you use exceptions more than totally exceptionally, according to your definition, then those pages will be loaded to memory and kept there – while the optimisation sounds very clever and sensible, I have a hard time imagining that exception metadata is huge in comparison with the rest of the program code. Once that data is in memory, you're back to the numbers in the article, where throwing an exception can take barely more than a microsecond. That's huge compared to a function call, but is fast compared to a lot of things that an overly cautious programmers wouldn't dare follow up with an exception. To steal your own example, checking whether a file exists certainly falls into this category – that's likely to take milliseconds. (Even your other example – no data available to read (I'm imagining a select / poll situation) – would probably be fine with exceptions, though I agree it's not the right tool for that situation. Just making a call to the OS is likely to cost a comparible amount to throwing an exception, even though you're describing a non blocking call. And by definition this won't end to being called in a critical inner loop - if data becomes available then the exception will do being thrown.)
- benibela 7y agoThe overhead also depends on how the compiler implements them Some compilers use setjump/longjump (sjlj), and then it needs to save all registers somewhere on every try{}. Probably also in every function that calls a destructor. That is very slow, even if no exception is thrown at runtime
- flqn 7y agoI can't say I'm a fan of these sorts of articles. "Which strategy is most performant?!" is rarely the appropriate measure for an error handling strategy, it's more nuanced than that. Concerns like what makes a recoverable error vs an unrecoverable one are more important. Ensuring correctness is more important. The usability of an API is more important. Readability and maintainability are more important. Knowing what errors mean in terms of your domain model is more important. Use what fits for your situation, don't take the "when all I have's a hammer..." attitude to anything. Sometimes return codes make sense (eg in hot-loop code). Sometimes exceptions make sense (in constructors, for example. And no, two-step initialisation is not a good alternative.). Sometimes neither make sense and you actually should just terminate the whole program (out-of-memory). Sometimes contracts make more sense (precondition checks, validating function arguments). Anyway, regardless of your strategy if you can provide some method of querying if a thing can succeed to your API, do so; it's always better to handle errors by just making sure you don't cause them!
- rumanator 7y agoI would add that a error-handling strategy should never be a concern wrt performance because obviously the purpose of this sort of code is to serve as guard rails and ideally never be called at all ever. If your error-handling code is triggered so often that you feel it might be causing a performance hit, you have far more serious problems than the performance overhead of how you handle errors.
- aothms 7y agoI mean, it depends, CGAL for example, which I'd consider a high quality software library, uses exceptions for their exact number type to redo the calculation using more precision when needed https://www.cgal.org/FAQ.html#uncertain_exception https://www.cgal.org/FAQ.html#uncertain_exception
- rumanator 7y agoYou're invoking the appeal to authority fallacy. Just because a popular package adopted a practice that does not make it right or good or acceptable. If that line of argument made any sense, you could argue that bugs present in CGAL represent best practices. Rule of thumb: if a code path is not exceptional then it should not be implemented with exceptions. The happy path is not an exception.
- Felk 7y agoIf you want to see Herb Sutter's take on exceptions (and RTTI), I can really recommend this talk where he explores his Result-esque (like in Rust) solution that he wrote a paper about and proposes for standardisation: https://youtu.be/ARYP83yNAWk https://youtu.be/ARYP83yNAWk
- not2b 7y agoIf exceptions aren't used, the error case has to be handled somehow. The usual approach is to return two results, and callers have to check the error flag somehow. If exceptions are very rarely thrown, then even with their very high overhead the total cost could well be lower than the cost of returning a separate error result and having callers check that result, perhaps by pattern matching.
- ncmncm 7y agoSpot the Howler: "it’s always best to catch an exception as close to the throw point as possible". No. It's fastest to do that, but if you cared about fast, you wouldn't be throwing at all. Exceptions, by definition, are off of the critical path. If there's a generalization to be made, it's that it's always best to catch them as far from the event as possible, because then you will have few catch clauses. If you have more than a few, you are Doing It Wrong. It shows a fundamental misunderstanding to worry about how fast throwing an exception is. You legitimately worry about how fast it is not to throw them, and that is generally quite fast indeed.
- m0zg 7y agoPerformance overhead, I'm not that worried about - if you're throwing exceptions routinely in the critical path, there's likely a lot else wrong with your code. The crappy part about C++ exceptions is that they also introduce a substantial binary size overhead.
- ec109685 7y agoAny idea of the cost of the final approach they used for the “success” case? E.g. if the rand errored 1/100, would the cost of returning that union object outweigh the exception?
- Thorrez 7y agoInstead of using Expected you could also use std::optional or std::variant. Expected might have a slightly clearer API for this use case though.
- LessDmesg 7y ago1. Exceptions are necessary for correctness. This alone solves the question of whether they should be used. Error codes just don't cut it because they only handle expected errors (which is never enough in the real world). 2. Arguing about C++ exception performance is pretty pointless anyway since C++ performance is its (only) strong side. Until Java/C# matches C++ in e.g. cutting edge 3D engines, this question won't even become interesting.
- sika_grr 7y agoI disagree with both points, in fact with every statement you made. Correctness can be achieved without use of exceptions (whether this is easier or harder to do is up for debate). Next, in some cases correctness may be sacrificed for performance reasons (e.g. solving a traveling salesman problem). I find it easier to properly handle unexpected error codes than unexpected exceptions. C++ has other strong sides beside performance. Java/C# often throw exceptions in very non-exceptional cases (e.g. .Net's File.Delete method).
- LessDmesg 7y agoNope, there's no practical way to cover all possible cases. There is always a possibility of a totally exceptional situation arising (we're only human). And then your beautiful ole error-cods returnin' function either takes down your whole application (which might result in someone dying while on life support in a hospital) or plods on with a successfull error code (leaving your program in an incorrect state). All because you didn't have a catch-all-exceptions clause. This is why I find C++'s "nothrow" a joke, by the way. It's not within human power to guarantee that any piece of code never throws exceptions. A gamma ray might flip a bit. You never know. The only practical way to reliability is the Erlang way: be prepared for errors arising anywhere anytime, not rely on fallible notions of what can and cannot cause errors.
- sika_grr 7y agoThere is a difference between preventing a program crash and handling errors. Please don't use catch-all-exceptions clause for either, especially not in hospital machines. The difference between writing if (status != SUCCESS) and catch (Exception) is that the former will allow the program to actually crash when it should (see panic in Rust). You should not be able to recover from OutOfMemory or GammaRayFlippedABit errors (use ECC memory to prevent that). You should restart the process (or the entire machine) because you can't hope the developer was wise enough to destroy and recreate appropriate amount of state. The additional problem with exceptions in Java and C# is that many times you don't care about failure of some operations, but the language practically forces you to write catch-all statement to prevent a program crash, so you end up accidentally catching unrecoverable errors.
- fwsgonzo 7y agoI made a website where you can measure these things in both time and instructions. It's not perfect because it's a VM, but at least you can see for yourself the horror show that is the first exception thrown. And also the bloat that exceptions add. I still use exceptions simply because they are part of the language, although sparingly. But I have a gut feeling that the implementation of C++ exceptions in the current compilers, and the ABI, is just subpar. It has to be, because nothing that isn't a context switch should be measured in hundreds of thousands of instructions. See: https://droplet.fwsnet.net https://droplet.fwsnet.net Just experiment with adding and removing exceptions, and compare it. It's shocking. Hello world using printf: Instruction count: 14988 Binary size: 345 KB Hello world try->throw->catch (and printf): Instruction count: 286926 Binary size: 397 KB Hello world using std::cout: Instruction count: 63502 Binary size: 888 KB
- ncmncm 7y agoYou are just measuring initialization time. The global "C" locale used by printf is statically initialized. With constexpr that could be done in a C++ library if anybody cared. Runtime spent initializing might matter for startup in an embedded system, but they tend not to use cout much. You could submit a PR constexpr-ing std::cout initialization.
- lanevorockz 7y agoIt's popular because it gives to people what they want to hear. There has been lots of research that gave different results and people just love to ignore.