11 ms·

Make CPython segfault in 5 lines of code

130 points by coolreader18 7y ago

angrygoat 7y ago
Here's the Python bug tracking the issue: https://bugs.python.org/issue39091 https://bugs.python.org/issue39091
sigjuice 7y ago
A quick search in bugs.python.org shows several crashes. I don’t know much about Python. Is there anything particularly special about this one?
- coolreader18 7y ago
  Not necessarily, but I just found it interesting that it wasn't really anything all that crazy - it could in theory be possible to encounter this in a big enough code base that makes heavy use of coroutines and custom exception types.
  - xapata 7y ago
    It is crazy to have a class constructor return something that isn't an instance of the class. That's nonsense code and is unlikely to occur in a codebase of any size, regardless of how often they define custom exception types. I wouldn't have bothered filing the bug.
    - saagarjha 7y ago
      The Python interpreter should not segfault for this.
      - xapata 7y ago
        Yes, of course. But is it worth your time to fix it? Probably not. If I jump on my bed enough, it'd probably break, but I'm not complaining to the manufacturer about the issue.
        serf 7y ago
        to re-analogize with tools : if I used my Snap-On(tm) wrench as a prybar (incorrect usage) and broke it, Snap-On would still replace it in exchange for the broken tool and knowledge of the situation that broke it. To pretend that a language bug isn't worth reporting because you and your codebases will never encounter it seems short-sighted. Down the line, years from now, who knows what you'll have to do to get something to work. Maybe it'll be something this silly, and you'll be happy that the folks before you encountered it and remediated it. All that said, from a practical standpoint I agree with you. If you're doing something wacky, and it turns out as wacky as you thought it would, you're probably attacking the problem from the wrong angle, anyway. I just want to remind everyone that 'wacky' things are required and implemented daily in codebases around the world -- regardless of how bad they smell.
- mentat 7y ago
  I read it as related to the "limit Python to 1 million lines so it can be stable."
  - OskarS 7y ago
    These two things have nothing to do with each other. The 1 million limit wouldn't have any effect here.
- lucy_gatenby 7y ago
  Just for fun, here's another one: import sys, threading def r(): sys.stdin.buffer.read(1) t = threading.Thread(target=r, daemon=True) t.start()
  - saagarjha 7y ago
    At least that one is an abort…
  - Thorrez 7y ago
    Here's a Python 2 segfault I ran into recently. import sys, threading, time t = threading.Thread(target=sys.stdin.read, args=(1,)) t.start() time.sleep(1) sys.stdin.close() Run it then after a few seconds press enter. It doesn't segfault in Python 3, but it still doesn't behave how I'd like, because I would like the close() to unblock the read(), but it doesn't unblock the read(), the read() still hangs until it gets some input.
    - loeg 7y ago
      The whole threading library in Python is a mess. Python was designed around single threaded programs with shared-nothing state and the cracks show as you move beyond that. The whole idea of replacing the GIL with... multiple same-process distinct-state Python interpreters with cheap-ish message passing sort of highlights how ugly it gets.
      - h2odragon 7y ago
        Back around python 1.5, there was almost a fork of python where every object had locks, there were memory arenas, and multiprocessing was almost thoughtlessly easy. That and stackless would've been great.
        PixyMisa 7y ago
        It was also terribly slow. IronPython did that too, on .Net. It ran around one quarter the speed of CPython.
lpghatguy 7y ago
Segfaults in scripting languages are remarkably common, especially if arbitrary bytecode can be loaded into the VM. One I ran into in the wild recently is that in older versions of Lua, exceptions in GC finalizers (the `__gc` metamethod) can trigger a segfault. In those same versions of Lua, the bytecode format is notoriously dangerous to load. I wonder whether this will be a large component of newer scripting language implementations. Do these safety issues warrant use of memory safe languages like Rust, or use of existing sandboxed VM implementations like WebAssembly?
- danielheath 7y ago
  IMO the biggest reason to adopt the WebAssembly format is that a segfault inside the runtime doesn't affect the host process at all. It's a plausible approach to a fully-safe, near-native-speed plugin architecture.
  - Thaxll 7y ago
    Segfault inside the runtime, What does it means exactly? A segfault is by definition at the OS level. WebAssembly koolaid is strong on HN, let's wait the first exploits that escapes the runtime to assess the "fully-safe" architecture.
    - wizzwizz4 7y ago
      I think they're trying to say that an out of bounds memory access within the emulated WebAssembly machine can be caught by the WebAssembly runtime. (I don't know whether this is true; I hardly know anything about WebAssembly.) The way they said it, though, makes it sound like WebAssembly is implemented with full process sandboxing or something, which is patently false. It works that way in neither Chrome nor Firefox, and there are no other browsers right now.
      - rini17 7y ago
        WebAssembly follows the unfortunate C paradigm that no checks are done at runtime, only these that programmer requests explicitly (and only if no undefined behavior is involved), to improve speed. The WASM sandbox can call only set of specified host functions, but I expect so much functionality snowballing inside sandboxes that we'll have to allow everything including unsafe ones, anyway.
paulddraper 7y ago
If # of lines are important, this problem can actually be demonstrated in 1 line: for x, x.__new__ in [(__import__('queue').Full, print)]: __import__('glob').iglob(0).throw(x)
- filmor 7y ago
  This doesn't trigger a segfault for me in Python 3.7, just an exception that `print_exception` expects an `Exception`. Someone commented below the gist with this one-liner: (i for i in []).throw(type('E', (BaseException,), dict(__new__=lambda cls, *args: cls))()) I managed to golf it a bit down to this ;): n="__new__";(i for i in []).throw(type(n,(IOError,),{n:lambda c,*a:c})())
  - ehsankia 7y ago
    Just one character but replace `[]` with `n` too :)
    - chrismorgan 7y ago
      Or save that character by removing the space before `[]` instead (which you can’t do if you write `n`).
  - paulddraper 7y ago
    Mine produces segfault with $ python3.7 -VV Python 3.7.5 (default, Nov 7 2019, 10:50:52) [GCC 8.3.0] Some more golfing with yours: (i for i in[]).throw(type('',(IOError,),{'__new__':lambda a,*b:a}))
dependenttypes 7y ago
This would not have happened in a language with a proper type system as the type checker would have rejected the program at compile time.
- saagarjha 7y ago
  Type confusion and memory corruption is still possible in statically-typed languages, generally due to bugs in the runtime.
- _ZeD_ 7y ago
  literally 3 seconds with google:"ghc segfault" https://github.com/lehins/ghc-segfault https://github.com/lehins/ghc-segfault
- shakna 7y ago
  > This would not have happened in a language with a proper type system as the type checker would have rejected the program at compile time. How about in Rust, then? [0] Bugs happen in every language. When memory corruption occurs, you can segfault. [0] https://users.rust-lang.org/t/rust-guarantees-no-segfaults-with-only-safe-code-but-it-segfaults-stack-overflow/4305 https://users.rust-lang.org/t/rust-guarantees-no-segfaults-w...
  - dependenttypes 7y ago
    I said "a language with a proper type system". Rust is not one such language.
    - Znafon 7y ago
      Can you give an example of such language so we don't have to guess? Apparently neither Rust, Haskell nor Go fit.
      - dependenttypes 7y ago
        check my username
mailslot 7y ago
wtf
Loranubi 7y ago
Even some "safe" python libraries can be segfaulted using similar techniques. See for example https://github.com/pydata/numexpr/issues/323 https://github.com/pydata/numexpr/issues/323 I think this particular bug is even marked 'wont fix'. But I cannot find the bpo at the moment.
loeg 7y ago
(CPython 3)
PixyMisa 7y ago
Works fine in PyPy, which is not surprising since the execution model is entirely different.
westurner 7y ago
FWIW, this segfaults CPython in 2 lines: import ctypes ctypes.cast(1, ctypes.py_object) Interestingly, this works: import ctypes, gc x = 22 _id = id(x) del x gc.collect() y = ctypes.cast(_id, ctypes.py_object).value assert y == 22

130 points by coolreader18 7y ago

Here's the Python bug tracking the issue: https://bugs.python.org/issue39091 https://bugs.python.org/issue39091

A quick search in bugs.python.org shows several crashes. I don’t know much about Python. Is there anything particularly special about this one?

coolreader18 7y ago
Not necessarily, but I just found it interesting that it wasn't really anything all that crazy - it could in theory be possible to encounter this in a big enough code base that makes heavy use of coroutines and custom exception types.
- xapata 7y ago
  It is crazy to have a class constructor return something that isn't an instance of the class. That's nonsense code and is unlikely to occur in a codebase of any size, regardless of how often they define custom exception types. I wouldn't have bothered filing the bug.
  - saagarjha 7y ago
    The Python interpreter should not segfault for this.
    - xapata 7y ago
      Yes, of course. But is it worth your time to fix it? Probably not. If I jump on my bed enough, it'd probably break, but I'm not complaining to the manufacturer about the issue.
      - serf 7y ago
        to re-analogize with tools : if I used my Snap-On(tm) wrench as a prybar (incorrect usage) and broke it, Snap-On would still replace it in exchange for the broken tool and knowledge of the situation that broke it. To pretend that a language bug isn't worth reporting because you and your codebases will never encounter it seems short-sighted. Down the line, years from now, who knows what you'll have to do to get something to work. Maybe it'll be something this silly, and you'll be happy that the folks before you encountered it and remediated it. All that said, from a practical standpoint I agree with you. If you're doing something wacky, and it turns out as wacky as you thought it would, you're probably attacking the problem from the wrong angle, anyway. I just want to remind everyone that 'wacky' things are required and implemented daily in codebases around the world -- regardless of how bad they smell.
mentat 7y ago
I read it as related to the "limit Python to 1 million lines so it can be stable."
- OskarS 7y ago
  These two things have nothing to do with each other. The 1 million limit wouldn't have any effect here.
lucy_gatenby 7y ago
Just for fun, here's another one: import sys, threading def r(): sys.stdin.buffer.read(1) t = threading.Thread(target=r, daemon=True) t.start()
- saagarjha 7y ago
  At least that one is an abort…
- Thorrez 7y ago
  Here's a Python 2 segfault I ran into recently. import sys, threading, time t = threading.Thread(target=sys.stdin.read, args=(1,)) t.start() time.sleep(1) sys.stdin.close() Run it then after a few seconds press enter. It doesn't segfault in Python 3, but it still doesn't behave how I'd like, because I would like the close() to unblock the read(), but it doesn't unblock the read(), the read() still hangs until it gets some input.
  - loeg 7y ago
    The whole threading library in Python is a mess. Python was designed around single threaded programs with shared-nothing state and the cracks show as you move beyond that. The whole idea of replacing the GIL with... multiple same-process distinct-state Python interpreters with cheap-ish message passing sort of highlights how ugly it gets.
    - h2odragon 7y ago
      Back around python 1.5, there was almost a fork of python where every object had locks, there were memory arenas, and multiprocessing was almost thoughtlessly easy. That and stackless would've been great.
      - PixyMisa 7y ago
        It was also terribly slow. IronPython did that too, on .Net. It ran around one quarter the speed of CPython.

lpghatguy 7y ago

Segfaults in scripting languages are remarkably common, especially if arbitrary bytecode can be loaded into the VM. One I ran into in the wild recently is that in older versions of Lua, exceptions in GC finalizers (the `__gc` metamethod) can trigger a segfault. In those same versions of Lua, the bytecode format is notoriously dangerous to load. I wonder whether this will be a large component of newer scripting language implementations. Do these safety issues warrant use of memory safe languages like Rust, or use of existing sandboxed VM implementations like WebAssembly?

danielheath 7y ago
IMO the biggest reason to adopt the WebAssembly format is that a segfault inside the runtime doesn't affect the host process at all. It's a plausible approach to a fully-safe, near-native-speed plugin architecture.
- Thaxll 7y ago
  Segfault inside the runtime, What does it means exactly? A segfault is by definition at the OS level. WebAssembly koolaid is strong on HN, let's wait the first exploits that escapes the runtime to assess the "fully-safe" architecture.
  - wizzwizz4 7y ago
    I think they're trying to say that an out of bounds memory access within the emulated WebAssembly machine can be caught by the WebAssembly runtime. (I don't know whether this is true; I hardly know anything about WebAssembly.) The way they said it, though, makes it sound like WebAssembly is implemented with full process sandboxing or something, which is patently false. It works that way in neither Chrome nor Firefox, and there are no other browsers right now.
    - rini17 7y ago
      WebAssembly follows the unfortunate C paradigm that no checks are done at runtime, only these that programmer requests explicitly (and only if no undefined behavior is involved), to improve speed. The WASM sandbox can call only set of specified host functions, but I expect so much functionality snowballing inside sandboxes that we'll have to allow everything including unsafe ones, anyway.

paulddraper 7y ago

If # of lines are important, this problem can actually be demonstrated in 1 line: for x, x.__new__ in [(__import__('queue').Full, print)]: __import__('glob').iglob(0).throw(x)

filmor 7y ago
This doesn't trigger a segfault for me in Python 3.7, just an exception that `print_exception` expects an `Exception`. Someone commented below the gist with this one-liner: (i for i in []).throw(type('E', (BaseException,), dict(__new__=lambda cls, *args: cls))()) I managed to golf it a bit down to this ;): n="__new__";(i for i in []).throw(type(n,(IOError,),{n:lambda c,*a:c})())
- ehsankia 7y ago
  Just one character but replace `[]` with `n` too :)
  - chrismorgan 7y ago
    Or save that character by removing the space before `[]` instead (which you can’t do if you write `n`).
- paulddraper 7y ago
  Mine produces segfault with $ python3.7 -VV Python 3.7.5 (default, Nov 7 2019, 10:50:52) [GCC 8.3.0] Some more golfing with yours: (i for i in[]).throw(type('',(IOError,),{'__new__':lambda a,*b:a}))

dependenttypes 7y ago

This would not have happened in a language with a proper type system as the type checker would have rejected the program at compile time.

saagarjha 7y ago
Type confusion and memory corruption is still possible in statically-typed languages, generally due to bugs in the runtime.
_ZeD_ 7y ago
literally 3 seconds with google:"ghc segfault" https://github.com/lehins/ghc-segfault https://github.com/lehins/ghc-segfault
shakna 7y ago
> This would not have happened in a language with a proper type system as the type checker would have rejected the program at compile time. How about in Rust, then? [0] Bugs happen in every language. When memory corruption occurs, you can segfault. [0] https://users.rust-lang.org/t/rust-guarantees-no-segfaults-with-only-safe-code-but-it-segfaults-stack-overflow/4305 https://users.rust-lang.org/t/rust-guarantees-no-segfaults-w...
- dependenttypes 7y ago
  I said "a language with a proper type system". Rust is not one such language.
  - Znafon 7y ago
    Can you give an example of such language so we don't have to guess? Apparently neither Rust, Haskell nor Go fit.
    - dependenttypes 7y ago
      check my username

mailslot 7y ago

wtf

Loranubi 7y ago

Even some "safe" python libraries can be segfaulted using similar techniques. See for example https://github.com/pydata/numexpr/issues/323 https://github.com/pydata/numexpr/issues/323 I think this particular bug is even marked 'wont fix'. But I cannot find the bpo at the moment.

loeg 7y ago

(CPython 3)

PixyMisa 7y ago

Works fine in PyPy, which is not surprising since the execution model is entirely different.

westurner 7y ago

FWIW, this segfaults CPython in 2 lines: import ctypes ctypes.cast(1, ctypes.py_object) Interestingly, this works: import ctypes, gc x = 22 _id = id(x) del x gc.collect() y = ctypes.cast(_id, ctypes.py_object).value assert y == 22