4 ms·

An interesting and non-obvious usecase: debugging python code. Standard debuggers use ptrace to implement breakpoints by replacing instructions in debugged pro

An interesting and non-obvious usecase: debugging python code. Standard debuggers use ptrace to implement breakpoints by replacing instructions in debugged process with the assembly code INT3 (or equivalents). This makes the cpu trap when it reaches that instruction which transfers control to the kernel who pauses the process and notifies the debugger. This approach doesn't work for Python because there are no native assembly instructions for the Python code - only for the C interpreter. Most Python debuggers attach to already running processes by injecting a few lines of native code (ptrace on Linux, CreateRemoteThread on Windows) which then bootstraps a new Python thread that actually implements the debugger. (As the linked article mentions, on Linux you can inject code using ptrace today but it isn't the most convenient.) This is also the trick that tools like pyrasite take to inject python code into running processes. On that note, if anyone wants to easily debug/profile code running on kubernetes clusters with no prior setup, I'm looking for beta testers for an open source framework my cofounder and I are developing. It enables general application maintenance on kubernetes and not just debugging/profiling, but that's the part we need testers for right now [1]: https://en.m.wikipedia.org/wiki/INT_(x86_instruction) https://en.m.wikipedia.org/wiki/INT_(x86_instruction)

aidos 6y ago
Thanks for that rundown. Super interesting.
phaedrus 6y ago
I implemented a debugger for an interpreted language (not Python, but similar design problem). I added a C "API" to the interpreter that was designed to be easily callable via GDB. IIRC it consisted of just three functions: "break on script line", "catch break here", and "why did I break?" My "debugger" was a shim or wrapper around GDB. For debugging C/C++ code, it just piped standard in/out with GDB. However if it recognized a "set breakpoint" that was actually for a line of script code, it would intercept it. Instead of telling GDB to set a breakpoint there, it would do two things: It would use GDB to manually call a function in the target program instructing the interpreter to watch for that script line ("set script breakpoint"). Then it would set a normal (native) breakpoint, but in the special function the interpreter would call from its main loop ("catch break here"). Conversely while relaying output from GDB back to the IDE, if it saw it hit a breakpoint in the "catch break here" function, it wouldn't report that. Instead the shim would use GDB to call the "why did I break" function and report the result of that back to the IDE. It was like magic getting to see the IDE bring up the correct script file and line as if the debugger hit it! Other extensions like single stepping are obvious from these primitives.
- bch 6y ago
  Curious to hear more about this

aidos 6y ago

Thanks for that rundown. Super interesting.

phaedrus 6y ago

I implemented a debugger for an interpreted language (not Python, but similar design problem). I added a C "API" to the interpreter that was designed to be easily callable via GDB. IIRC it consisted of just three functions: "break on script line", "catch break here", and "why did I break?" My "debugger" was a shim or wrapper around GDB. For debugging C/C++ code, it just piped standard in/out with GDB. However if it recognized a "set breakpoint" that was actually for a line of script code, it would intercept it. Instead of telling GDB to set a breakpoint there, it would do two things: It would use GDB to manually call a function in the target program instructing the interpreter to watch for that script line ("set script breakpoint"). Then it would set a normal (native) breakpoint, but in the special function the interpreter would call from its main loop ("catch break here"). Conversely while relaying output from GDB back to the IDE, if it saw it hit a breakpoint in the "catch break here" function, it wouldn't report that. Instead the shim would use GDB to call the "why did I break" function and report the result of that back to the IDE. It was like magic getting to see the IDE bring up the correct script file and line as if the debugger hit it! Other extensions like single stepping are obvious from these primitives.

bch 6y ago
Curious to hear more about this