2 ms·
Speaking of GDB and jvns, how come there's no good out of process profiler for Python? http://jvns.ca/blog/2016/06/12/a-weird-system-call-process-vm-readv/ htt
by pas 10y ago
Speaking of GDB and jvns, how come there's no good out of process profiler for Python?
http://jvns.ca/blog/2016/06/12/a-weird-system-call-process-vm-readv/ http://jvns.ca/blog/2016/06/12/a-weird-system-call-process-v...
I love the JVM's easily trace-ability, though that involves safepoints, so that's not completely out of process either.
- wyldfire 10y agoPuDB is really good IMO. But I suppose it may not qualify as out-of-process. In practice, I don't end up needing it anywhere near as often as I need gdb for c/c++, but it's handy to have just in case.
- jlouis 10y agoPure guesswork: Java stems from Sun. Sun made Solaris. And Solaris had an ideology about transparency and discoverability of running systems. They have some outright amazing tools for introspection into a binary that is running. This probably creates an environment in which the same would happen for the JVM. In fact, the `jstack` name seem very familiar to DTrace (and mdb(1)) users. Days since I last solved a problem on the JVM through a heapDump: 2.
- jvns 10y agoI'd love to know the answer to this.
- filereaper 10y agoDepends on how the language runtime defines its calling convention, does it follow System Linkage [1] or does it implement its own internal linkage ie Private Linkage? If it follows standard System Linkage, its easy to point gdb or any other system debugger or profiler to debug and profile the application. Some runtimes have a mix of System and Private linkage, ie C functions will follow System Linkage but JIT'ed code frames might follow private linkage. This makes for difficult stack-walking by system native debuggers and profilers. You'd have to teach GDB via an extension how to walk the non-standard frames. So yea, long story short, it depends on the linkage convention the implementers of the language runtime decided to follow. [1] http://www.x86-64.org/documentation/abi.pdf http://www.x86-64.org/documentation/abi.pdf
- pas 10y agoI mean, GDB already has good scripts to extract CPython stuff from inferior processes, but there's no GDB API so that anyone could make a profiler out of it. At least I haven't found anything. (So GDB's scriptability seems to be very restricted.)
- emfree 10y agoGreat question. I wondered the same thing a while ago, and tried to build one using SystemTap (https://github.com/emfree/pystap https://github.com/emfree/pystap). Couple reasons why this isn't too easy: * "Python" in general might mean you're on Linux/Windows/whatever, and it might mean CPython, PyPy, or some other runtime. But any out-of-process instrumentation is gonna have to be pretty platform/runtime specific. * Even if we restrict ourselves to, say, CPython on Linux, the interpreter's internals aren't super friendly to this sort of inspection from the outside. You have to rely on and also work around implementation details. Example: to get a Python call stack, you want to look at `PyThreadState_Current` (basically the same idea as `ruby_current_thread` in that excellent linked post of Julia's, I think). But this happens to be null whenever the GIL is released, e.g. when doing network I/O, and then you're kind of out of luck. So you'll already have trouble usefully profiling a single-threaded I/O-intensive program. * Oh and you pretty much need debug symbols in your CPython binary (I think? Tell me if this isn't true!). Most production CPython builds don't have them. So you have to get the right binary, and rebuild any application dependencies with C extensions. Not hard but annoying. There is potential though! With some work, we definitely could have a better story for out-of-process Python profiling a la Linux perf.