7 ms·

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raisi

• coldtea 5y ago
  >Because exceptions are a fundamental control-flow mechanism in Python (unlike in C++, where they should only be used for exceptional control flow), I'm not sure if there will be a net benefit to "zero cost" exceptions. Well, they might be "a fundamental control-flow mechanism" but for every StopIteration (for an example of a control flow exception use), there are multiple (up to millions) of traversed elements that didn't throw an exception.
• mplanchard 5y ago
  This was really informative, thanks! Also “it’s just like sufficiently advanced technology” made my morning.
• zeotroph 5y ago
  Note that the generated instructions - even if they are never run as in [1] - still consume cache and might hinder further optimizations. Which is why `noexcept` is becoming more popular. And because there is no GC, code must be written to be exception-safe in all conditions which is often forgotten. 1: https://godbolt.org/z/bKfG14P64 https://godbolt.org/z/bKfG14P64 - the difference between `e()` and `n()` is that one is marked `noexcept`. Both `f()` and `g()` run directly to `ret` (return) in the normal case.
  • vlovich123 5y ago
    Isn’t exception handling code placed in the cold section of the binary these days so that the impact on cache is nonexistent? Yup, the generated assembly does this. There’s some minimal extra instructions still to setup the frame but the bulk of the exception handling lives elsewhere.
  • bregma 5y ago
    Yeah, noexcept(true) is identical to wrapping the function with a try-catch construct in which the catch clause simply calls std::terminate(). Your godbolt example doesn't include the definition of n() so it doesn't show that. Adding noexcept(true) has a cost (because it's a kind of exception handling) but also allows the compiler to optimize out some of that cost under some circumstances. Nevertheless, an example that demonstrates the exception path doesn't say much about the "zero-cost" non-exception path. In C++ exception handling is expensive. Outside of actually directly handling exceptions they have zero cost. There are no generated instructions. They do not consume cache. They do not hinder further optimizations.
    • zeotroph 5y ago
      Indeed, in the following case of a 4-deep call stack, each with its own exception, the bulk of the handling code can be moved elsewhere (already marked as cold by the compiler), but nonetheless there are a few instructions which won't matter in most cases but are still required to jump there and thus end up in the instruction cache. https://godbolt.org/z/eh1d4K1M7 https://godbolt.org/z/eh1d4K1M7
  • 5y ago
• Someone 5y ago
  One of the stated benefits of this is > Calls to Python functions would be faster as frame objects would be considerably smaller. Currently each frame carries 240 bytes of overhead for exception handling. I guess that’s where this will pay even in a language/ecosystem where exceptions aren’t exceptional.
  • bregma 5y ago
    It's a time-space tradeoff. A few bytes less overhead on the stack, a few CPU cycles more overhead when raising an exception. Is it a net win? Show me empirical test results.
• formerly_proven 5y ago
  Excellent post! Python Bytecode is a little more naive / high level than this though, so stuff like exceptions and their nested handlers are actually implemented in the VM itself. The VM has essentially a second stack for exception and context manager blocks. The compiled bytecode essentially looks like this: SETUP_TRY 10 # address where exceptions will be handled some stuff that might explode POP_BLOCK JUMP 20 # jump over exception handlers LOAD exception type # address 10 COMPARE # check if exception type matches ... handler for the type stuff after the try-except # address 20 The corresponding source would look like try: some stuff that might explode except exception type: ... handler for the type stuff after the try-except Python's bytecode compiler is generally a 1:1 translation of the AST; it never optimizes, e.g: [value for value in list] Translates to something like LIST_NEW FOR_EACH STORE LOAD LIST_APPEND JUMP BACK Note how "value" generated a store-load pair. The VM checks whether PyErr (pointer to exception) is set after basically everything. Similarly, extension modules check PyErr after every call into the interpreter, e.g. PyObject *attr = PyObject_GetAttr(someattr, somepystr); if(PyErr_Occurred()) { // or !attr // handle exception } This gets old pretty fast.
  • mixedmath 5y ago
    I'm growing more interested in actually understanding some of the internals that you mention here. I know this is a bit of a tangent, but is there a better way to approach understanding python's internals than reading the source (which feels a bit monolithic to me right now)?
    • formerly_proven 5y ago
      The central dispatch of the VM is a good place to start: https://github.com/python/cpython/blob/main/Python/ceval.c#L1678 https://github.com/python/cpython/blob/main/Python/ceval.c#L...
    • boothby 5y ago
      I thought I had a pretty good handle on Python internals, until some time early this year when I took an interest in the generated bytecode. I'd read plenty of the cpython source, written lots of cython extensions, etc., but somehow missed the middle piece. Fortunately, it's really easy to get at the bytecode, and quite instructive. Random inquiry: how do generator functions work? In [1]: def foo(): ...: yield from range(10) ...: In [2]: import dis In [3]: dis.dis(foo) 2 0 LOAD_GLOBAL 0 (range) 2 LOAD_CONST 1 (10) 4 CALL_FUNCTION 1 6 GET_YIELD_FROM_ITER 8 LOAD_CONST 0 (None) 10 YIELD_FROM 12 POP_TOP 14 LOAD_CONST 16 RETURN_VALUE From there, you can read how each of those bytecode instructions is implemented in ceval.c, which formerly_proven links to. edit: probably nice to have the actual disassembly of a list comprehension, too: In [4]: def bar(): ...: return [x for x in range(10)] ...: In [5]: dis.dis(bar) 2 0 LOAD_CONST 1 (<code object <listcomp> at 0x7f814059f450, file "<ipython-input-4-a7a3cc7e7d7f>", line 2>) 2 LOAD_CONST 2 ('bar.<locals>.<listcomp>') 4 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (range) 8 LOAD_CONST 3 (10) 10 CALL_FUNCTION 1 12 GET_ITER 14 CALL_FUNCTION 1 16 RETURN_VALUE Disassembly of <code object <listcomp> at 0x7f814059f450, file "<ipython-input-4-a7a3cc7e7d7f>", line 2>: 2 0 BUILD_LIST 0 2 LOAD_FAST 0 (.0) >> 4 FOR_ITER 8 (to 14) 6 STORE_FAST 1 (x) 8 LOAD_FAST 1 (x) 10 LIST_APPEND 2 12 JUMP_ABSOLUTE 4 >> 14 RETURN_VALUE
• albertzeyer 5y ago
  > exceptions are a fundamental control-flow mechanism in Python Where do you have this from? In almost all Python code I have seen, exceptions are still for, as the name says, exceptions. So in the common cases, there would be no exceptions raised. I would assume that the exception handling code (under `except ...`) will be run only a tiny fraction of times compared to the other code, at least in most cases. I would argue, if one abuses exceptions for any sort of control flow logic, this is bad design. See also the list of builtin exceptions: https://docs.python.org/3/library/exceptions.html https://docs.python.org/3/library/exceptions.html From those, yes, there is StopIteration and StopAsyncIteration, which are used for control-flow, but the handling of those is anyway internal in CPython, and so the discussion about zero cost does not apply, as it would not change this (as far as I understand the current proposal). Otherwise, all other exceptions are not used for control-flow.
  • BiteCode_dev 5y ago
    The for loop in python is a try/except catching StopIteration in a trench coat. Also, EAFP, and the context manager protocol.
    • fouric 5y ago
      > The for loop in python is a try/except catching StopIteration in a trench coat. That's...horrifying, both from a computational efficiency perspective (due to the overhead of exceptions) and from a "beauty" perspective (exceptions are supposed to be exceptional, darn it!). Why is it implemented this way? Why not just use a normal loop and check a return value from the iterator?
      • dragonwriter 5y ago
        > Why not just use a normal loop and check a return value from the iterator? Because an iterator needs to be able to return any python value, and if you did that, there would be at least one python value that an iterator could not actually return. Unless you required the iterator to wrap real return values in a container (basically, an optional/maybe monad) but then you have unwrapping overhead on every real return value, which is worse.
• Diggsey 5y ago
  Also, it's not really the case that exceptions (when not thrown) are zero-cost in C++, and not because of the instruction cache or increase to static data size. The cost is that exception-throwing functions inhibit many of the optimizations performed by compilers, so they generate worse code, even though no "extra" code is actually executed.
  • bregma 5y ago
    Do they? I spend a lot of time dealing with the optimizers in GCC and haven't noticed that happening. In GCC, a throw is implemented as a single function call to the C++ runtime function __cxa_throw(). If a simple single function call will pessimize performance we're screwed as a species.
    • lgg 5y ago
      It does because calling that function adds a new control flow edge out of the block it is in, which means you can no longer prove certain cleanup code can safely be elided. Additionally the fact that C++ does not annotate whether a function throws in its signature that means you need to generate unwind data for most non-leaf functions since (short of whole program LTO) there is no way to safely to know that the functions they call don't throw. That means the size of the metadata necessary to support unwinding grows with the size of your binary, not the portion of the code using exceptions. Finally, C++ exceptions are dynamic, which imposes a higher cost when they are actually used than a static exception model. In general I suspect that explicit error handling (as is done in the Swift ABI) would result in better code because it would make the control flow edges more optimizable and optimizer could safely assume many functions don't ever need to be unwound. It would certainly make the binaries smaller due to the reduction in size of unwind tables (although some people including Stroustrop[1] argue there are ways to do similar optimizations to elide unwind data with existing C++ compilers, but I am not sure if anyone has ever built such an optimizer). There is some discussion about adding a static exception model (with annotated functions) described in P0709[2]. Various parts of that are controversial, though I hope the committee eventually finds a way to agree to some variant of it, because it would allow unification with other languages (it is semantically equivalent to Swift Error handling), and there are even C proposals that would be interoperable[3]. Full disclosure, I work on a dynamic linker written in C++, so I spend all day long writing C++ code that is built without exceptions or most of the standard library, but is part of the runtime machine that enables exception handling. [1]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1947r0.pdf http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p194... [2]: http://open-std.org/JTC1/SC22/WG21/docs/papers/2019/p0709r4.pdf http://open-std.org/JTC1/SC22/WG21/docs/papers/2019/p0709r4.... [3]: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2429.pdf http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2429.pdf
• sonthonax 5y ago
  What’s a DSO?
  • mjw1007 5y ago
    Dynamic Shared Object (in this context, probably a .so file).
    • sonthonax 5y ago
      Ah!
  • wahern 5y ago
    Dynamic shared object, or what's usually understood generically (in non-language specific terms) as a [binary] module or in GUI apps a plugin--something loaded and linked dynamically by the program, not by the linker at compile time or at program startup. In the Unix/C ecosystem "DSO" is a common term to describe binary modules, without reference to specific binary formats like ELF or mach-O. Especially in the land of Unix where dynamic linking has become very sophisticated and automated, many of these terms are losing their distinctive meanings as the various ways to link and load converge on the same underlying mechanisms which you can mix-and-match independently. But normally the difference between a DSO and a shared library is that shared library symbols (functions, global variables) are by default visible globally (at least on ELF; mach-O story is more complex), making them implicitly visible to subsequently loaded code, whether a shared library or DSO. On the other hand, you usually want all DSO symbols to remain private to the DSO by default, requiring the application at runtime to explicitly request a pointer to specific symbols as different DSOs might have symbols of the same name (especially true for plugins), leaving the application logic to decide which to use and when. Also, especially pertinent to Lua, Perl, Python, Ruby, and similar languages, a DSO usually implicitly relies on the main program to have loaded and exported certain core APIs, rather than the DSO explicitly linking to a shared library to provide them. (This still often requires special compile-time flags when building the DSO, and possibly also special flags to the main binary.) That ensures the VM/interpreter/engine and all DSOs are always calling into the same core runtime implementation, and ensures the main binary controls which that is. DSOs do often dynamically link to other shared libraries, but because of default symbol visibility semantics this can cause problems, such as two DSOs or a DSO and the main program compiled against different releases of OpenSSL with incompatible ABIs. (If you can manage to load two different version of OpenSSL, you have the reverse issue of accidentally passing objects between the two implementations.) Mitigations and resolutions for that problem differ between ELF-based and mach-O-based systems. (AIX is an outlier, which like Windows uses a PE-derived binary format that is significantly different than ELF or mach-O. Many of the semantic distinctions and jargon that developed around ELF don't make much sense in the context of PE.)
• omegalulw 5y ago
  Thanks for the informative post. > then the stack is unwound by interpreting the function table contents to step through the various CPU states in reverse order Why do you need the exception function table contents for this. Presumably, you only want to destroy stack objects so isn't the call stuck necessary and sufficient to unroll all the way back to the function with the catch handler? Also what are your thoughts on returning errors via a status vs exceptions. I love the former - they force you to somehow handle each error (pass it up or deal with it) vs exceptions where there is no accountability for a method that causally caused the exception.
  • bregma 5y ago
    You need to restore some execution context before invoking the destructors. The value of the implicit "this" parameter, for example, has to be set correctly before the call to the destructor. Depending on the CPU and calling conventions, other registers may need their values contextually restored. Then there's the stack crawling itself. Again, depending on the CPU and calling conventions, just finding where the return value is stored for a function can require some gymnastics (aarch64 I'm looking at you here). As to returning and handling errors via status returns instead of exceptions: you're factoring the cost of error handling into the good path. It's the opposite of zero-cost. I've never understood the argument that every piece of code needs to be able to handle every error that every subroutine it calls could ever encounter. No one even does that, and the end result is that "handle error by return codes" is usually faster than using exceptions, because the error handling ends up being not done at all. At least, that's my experience over 40 years of maintaining other people's code. As to the performance of languages that force error status returns to be dealt with vs. exception handling? Profile. Get some numbers. Examine the bias of the person generating the numbers.