4 ms·

You won't have a good time trying to debug a JVM or Node JS runtime with gdb. There are too many threads, too many moving parts, and trying to stop the world to

You won't have a good time trying to debug a JVM or Node JS runtime with gdb. There are too many threads, too many moving parts, and trying to stop the world to make sense of it is too invasive; you'll muck with timings and waits and timeouts and GC thresholds and all sorts, and your effort to figure out what's going on will have changed what's going on. And that's if gdb doesn't crash. IMO not enough people download and build the source. It's primitive, but adding logging or printfs to existing source, combined with strace if necessary, can be effective in a brutal way, where gdb - or any debugger - wouldn't get you far.

sgc 6y ago
I've always wondered if there shouldn't be a VSCode or other plugin that allows you to automatically throw in a bunch of printfs up a dependency chain to dump vars etc, and remove them just as easily.
- usefulcat 6y ago
  I'm not sure that would be as useful as it sounds. A big part of what you get from manually adding printfs is familiarity with the code you're instrumenting.
- user5994461 6y ago
  that's called a debugger, put a breakpoint somewhere, the application will pause right there and show you all the variables.
  - barrkel 6y ago
    Funny thing about debuggers is that they seem to encourage stepping - step in, step out, step over, single step, step to return. Stepping is an enormous waste of time with a debugger. Breakpoints are where it's at. Level up your breakpoints: logging breakpoints, conditional breakpoints, memory breakpoints (super handy when combined with deterministic memory allocation), syscall / library breakpoints - and soon you're going somewhere useful. Debugging is about formulating theories and proving them wrong, or right. Stepping doesn't scale; breakpoints can scale, if you know where to put them. I'm not sure that the gap between a debugger with breakpoints, and printfs in source, is that large, though. It's easier to write complex conditionals or log the state of a complex predicate in the source language than in whatever the debugger gives you - even if it gives you the native language, you typically have to write it all on a single line. And you need to dive deep into the source to understand where to put those breakpoints - or printfs. An underused debugging tool is an instrumenting profiler. The captured call graph is a useful clue for dynamic & indirect control flow in particular, without needing to decipher it from stepping or infer it from chasing source references and indirections. Things like generic message dispatchers and event callbacks are horrible to try and step through and aren't much better when eyeballing the source.
    - saagarjha 6y ago
      FWIW, I rarely put debugging logs in my code-almost always in cases where I can't stick a breakpoint on something because I can't get a debugger on it. 99% of your debugging printfs can be breakpoint commands. They're actually better because they are separate from your source code, so you aren't going to accidentally commit them or anything, they can change dynamically without recompiling or even re-running, and they have access to a lot more information than your compiler may let you print out.
      - barrkel 6y ago
        It's generally hard to add breakpoints in production. OTOH most logging systems aren't set up for dynamic contextual reconfiguration, a place could have a competitive advantage over debuggers with a bit of effort. IME I find it easier to deal with modifications to code than configuration in IDEs, precisely because I can identify diffs and if desired commit the modifications to source control on my own branch. I frequently debug things like third-party libraries (e.g. Ruby gems by adding my rvm gemset) to a git repo to track my modifications. It's generally harder to export and import breakpoints from an IDE. They are usually driven by line numbers, which aren't particularly stable over code changes.
zbentley 6y ago
While it's certainly much harder in managed runtimes, I've been quite happy with gdb's "where" output on JNI-bound or syscall-stuck JVM cores, and jstack (and more modern friends) where that didn't work. For my other HLL of choice, pyrasite[1] works quite well. When using all of those tools, I encountered significant out-of-the-box usability issues until I learned what those debugger-esque tools were doing with regards to their "target" programs' (or their postmortem heaps) stacks, signal handlers, and allocators. Combining the understanding of the latter concepts with the former tools is a killer combo, and, while I can't speak directly to your counterexample of the NodeJS runtime, I would be extremely surprised if tooling did not exist to perform similar debugging tasks with it. 1. https://github.com/lmacken/pyrasite https://github.com/lmacken/pyrasite
TeMPOraL 6y ago
Doesn't mean you have to be afraid of debugging your runtime. May be too much noise, or too much info, but sometimes getting a crashing runtime to dump core may be enough to clue you in on what's going wrong. The other day I did exactly that with JVM to discover why our JavaFX app crashes to desktop on customer's machine. Going through the crash dump, I immediately spotted DirectX calls being prominently displayed (IIRC top of one of the stack frames), and upon investigation, it turned out that the computer haven't had its graphics drivers updated in a long, long time.
tonyarkles 6y ago
Thank you for a reminder of one of my favourite debugging stories. I was working on an x86 bootloader and something was going wrong when making the transition into 32-bit protected mode. Ultimately I found the problem by... adding printfs inside qemu and recompiling, so it would output the CPU state registers for every instruction it was executing (luckily not too many to run before encountering the bug).

sgc 6y ago

I've always wondered if there shouldn't be a VSCode or other plugin that allows you to automatically throw in a bunch of printfs up a dependency chain to dump vars etc, and remove them just as easily.

usefulcat 6y ago
I'm not sure that would be as useful as it sounds. A big part of what you get from manually adding printfs is familiarity with the code you're instrumenting.
user5994461 6y ago
that's called a debugger, put a breakpoint somewhere, the application will pause right there and show you all the variables.
- barrkel 6y ago
  Funny thing about debuggers is that they seem to encourage stepping - step in, step out, step over, single step, step to return. Stepping is an enormous waste of time with a debugger. Breakpoints are where it's at. Level up your breakpoints: logging breakpoints, conditional breakpoints, memory breakpoints (super handy when combined with deterministic memory allocation), syscall / library breakpoints - and soon you're going somewhere useful. Debugging is about formulating theories and proving them wrong, or right. Stepping doesn't scale; breakpoints can scale, if you know where to put them. I'm not sure that the gap between a debugger with breakpoints, and printfs in source, is that large, though. It's easier to write complex conditionals or log the state of a complex predicate in the source language than in whatever the debugger gives you - even if it gives you the native language, you typically have to write it all on a single line. And you need to dive deep into the source to understand where to put those breakpoints - or printfs. An underused debugging tool is an instrumenting profiler. The captured call graph is a useful clue for dynamic & indirect control flow in particular, without needing to decipher it from stepping or infer it from chasing source references and indirections. Things like generic message dispatchers and event callbacks are horrible to try and step through and aren't much better when eyeballing the source.
  - saagarjha 6y ago
    FWIW, I rarely put debugging logs in my code-almost always in cases where I can't stick a breakpoint on something because I can't get a debugger on it. 99% of your debugging printfs can be breakpoint commands. They're actually better because they are separate from your source code, so you aren't going to accidentally commit them or anything, they can change dynamically without recompiling or even re-running, and they have access to a lot more information than your compiler may let you print out.
    - barrkel 6y ago
      It's generally hard to add breakpoints in production. OTOH most logging systems aren't set up for dynamic contextual reconfiguration, a place could have a competitive advantage over debuggers with a bit of effort. IME I find it easier to deal with modifications to code than configuration in IDEs, precisely because I can identify diffs and if desired commit the modifications to source control on my own branch. I frequently debug things like third-party libraries (e.g. Ruby gems by adding my rvm gemset) to a git repo to track my modifications. It's generally harder to export and import breakpoints from an IDE. They are usually driven by line numbers, which aren't particularly stable over code changes.

zbentley 6y ago

While it's certainly much harder in managed runtimes, I've been quite happy with gdb's "where" output on JNI-bound or syscall-stuck JVM cores, and jstack (and more modern friends) where that didn't work. For my other HLL of choice, pyrasite[1] works quite well. When using all of those tools, I encountered significant out-of-the-box usability issues until I learned what those debugger-esque tools were doing with regards to their "target" programs' (or their postmortem heaps) stacks, signal handlers, and allocators. Combining the understanding of the latter concepts with the former tools is a killer combo, and, while I can't speak directly to your counterexample of the NodeJS runtime, I would be extremely surprised if tooling did not exist to perform similar debugging tasks with it. 1. https://github.com/lmacken/pyrasite https://github.com/lmacken/pyrasite

TeMPOraL 6y ago

Doesn't mean you have to be afraid of debugging your runtime. May be too much noise, or too much info, but sometimes getting a crashing runtime to dump core may be enough to clue you in on what's going wrong. The other day I did exactly that with JVM to discover why our JavaFX app crashes to desktop on customer's machine. Going through the crash dump, I immediately spotted DirectX calls being prominently displayed (IIRC top of one of the stack frames), and upon investigation, it turned out that the computer haven't had its graphics drivers updated in a long, long time.

tonyarkles 6y ago

Thank you for a reminder of one of my favourite debugging stories. I was working on an x86 bootloader and something was going wrong when making the transition into 32-bit protected mode. Ultimately I found the problem by... adding printfs inside qemu and recompiling, so it would output the CPU state registers for every instruction it was executing (luckily not too many to run before encountering the bug).