4 ms·

Writing a Linux Debugger (2017)

108 points by lrsjng 3y ago

Veserv 3y ago
As someone who does debugger development as part of their day job, I can attest that this is a very good introduction to the details of basic debugger operation and implementation. If you are unfamiliar with the topic, I recommend you give it a try as it immediately starts giving useful nuggets of knowledge that are applicable in other circumstances if you are ever interested in doing low level or embedded development.
- HexDecOctBin 3y ago
  However, note that the libelfin library that the author uses is pretty much unmaintained. I am not sure what would be the right way to parse ELF and DWARF? libelf/libdwarf, libdw, or something else? Also, since you are probably in the know, is liblldb "extensible", in the sense that can liblldb can be used as a foundation of a new debugger with experimental features added on top of it?
  - msla 3y ago
    libbfd is apparently what the GNU debugger uses: https://ftp.gnu.org/old-gnu/Manuals/bfd-2.9.1/html_chapter/bfd_1.html https://ftp.gnu.org/old-gnu/Manuals/bfd-2.9.1/html_chapter/b... https://en.wikipedia.org/wiki/Binary_File_Descriptor_library https://en.wikipedia.org/wiki/Binary_File_Descriptor_library
    - tromey 3y ago
      gdb uses BFD for object file access but has its own DWARF reader.
  - chc4 3y ago
    The Rust goblin and gimli crates are very good, I've used them for ELF symbol resolution things before and they're a dream compared to libelf.
    - jcranmer 3y ago
      gimli works well enough for actually parsing the DWARF format, but it abstracts over somewhat less than what you would hope (in particular, you get to do all the fun stuff of figuring out where debugging information is yourself--which gets tedious when you want to support things like split-dwarf or debug index).
  - xvilka 3y ago
    libdwarf is the most complete C library, Rust options are also quite good. We already support both DWARF and PDB in Rizin[1] but working hard on improving that support[2] [1] https://github.com/rizinorg/rizin/tree/dev/librz/bin https://github.com/rizinorg/rizin/tree/dev/librz/bin [2] https://rizin.re/posts/gsoc-2023-announcement/#billow-debug-information-handling-improvements https://rizin.re/posts/gsoc-2023-announcement/#billow-debug-...
PhilipRoman 3y ago
Nice, I really like that it describes the DWARF format instead of just handwaving around it. I've been considering adding DWARF support to my compiler so that I can use mainstream debuggers with it and this article has just destroyed my last excuse to procrastinate. I must admit, the title got me very excited (it's still a cool article regardless) - something I've always wanted is a system-wide debugger for Linux. Something that can put breakpoints in arbitrary executables or shell scripts (via custom /bin/sh wrapper?) system wide, not just within a debugging session. I'd love to be able to put conditional breakpoints on, say, any process that opens a named pipe matching some regex. I find that when dealing with complex systems involving systemd services, cron jobs, udev and 30 different daemons running in background, gdb and strace simply aren't enough.
- jjnoakes 3y ago
  SystemTap or ebpf might be the way to go for visibility at that level. Not quite a debugger, but they can be very helpful.
  - PhilipRoman 3y ago
    I had heard of SystemTap, but had no idea it was that powerful. In particular, the user space marker feature sounds like what I need. Although it sounds like it may require manual instrumentation.
- schemescape 3y ago
  Not sure about Linux specifically, but it sounds like you’re looking for a kernel debugger that can communicate with a user mode debugger (or debuggers). Edit: actually SystemTap mentioned in a sibling comment sounds simpler if you just need instrumentation.
hoten 3y ago
Appreciated the How is Babby Formed reference in chapter two:)
setheron 3y ago
How does one debug the Linux kernel ? I want to step through some portions. Even with UML it sounds arduous

108 points by lrsjng 3y ago

As someone who does debugger development as part of their day job, I can attest that this is a very good introduction to the details of basic debugger operation and implementation. If you are unfamiliar with the topic, I recommend you give it a try as it immediately starts giving useful nuggets of knowledge that are applicable in other circumstances if you are ever interested in doing low level or embedded development.

HexDecOctBin 3y ago
However, note that the libelfin library that the author uses is pretty much unmaintained. I am not sure what would be the right way to parse ELF and DWARF? libelf/libdwarf, libdw, or something else? Also, since you are probably in the know, is liblldb "extensible", in the sense that can liblldb can be used as a foundation of a new debugger with experimental features added on top of it?
- msla 3y ago
  libbfd is apparently what the GNU debugger uses: https://ftp.gnu.org/old-gnu/Manuals/bfd-2.9.1/html_chapter/bfd_1.html https://ftp.gnu.org/old-gnu/Manuals/bfd-2.9.1/html_chapter/b... https://en.wikipedia.org/wiki/Binary_File_Descriptor_library https://en.wikipedia.org/wiki/Binary_File_Descriptor_library
  - tromey 3y ago
    gdb uses BFD for object file access but has its own DWARF reader.
- chc4 3y ago
  The Rust goblin and gimli crates are very good, I've used them for ELF symbol resolution things before and they're a dream compared to libelf.
  - jcranmer 3y ago
    gimli works well enough for actually parsing the DWARF format, but it abstracts over somewhat less than what you would hope (in particular, you get to do all the fun stuff of figuring out where debugging information is yourself--which gets tedious when you want to support things like split-dwarf or debug index).
- xvilka 3y ago
  libdwarf is the most complete C library, Rust options are also quite good. We already support both DWARF and PDB in Rizin[1] but working hard on improving that support[2] [1] https://github.com/rizinorg/rizin/tree/dev/librz/bin https://github.com/rizinorg/rizin/tree/dev/librz/bin [2] https://rizin.re/posts/gsoc-2023-announcement/#billow-debug-information-handling-improvements https://rizin.re/posts/gsoc-2023-announcement/#billow-debug-...

PhilipRoman 3y ago

Nice, I really like that it describes the DWARF format instead of just handwaving around it. I've been considering adding DWARF support to my compiler so that I can use mainstream debuggers with it and this article has just destroyed my last excuse to procrastinate. I must admit, the title got me very excited (it's still a cool article regardless) - something I've always wanted is a system-wide debugger for Linux. Something that can put breakpoints in arbitrary executables or shell scripts (via custom /bin/sh wrapper?) system wide, not just within a debugging session. I'd love to be able to put conditional breakpoints on, say, any process that opens a named pipe matching some regex. I find that when dealing with complex systems involving systemd services, cron jobs, udev and 30 different daemons running in background, gdb and strace simply aren't enough.

jjnoakes 3y ago
SystemTap or ebpf might be the way to go for visibility at that level. Not quite a debugger, but they can be very helpful.
- PhilipRoman 3y ago
  I had heard of SystemTap, but had no idea it was that powerful. In particular, the user space marker feature sounds like what I need. Although it sounds like it may require manual instrumentation.
schemescape 3y ago
Not sure about Linux specifically, but it sounds like you’re looking for a kernel debugger that can communicate with a user mode debugger (or debuggers). Edit: actually SystemTap mentioned in a sibling comment sounds simpler if you just need instrumentation.

hoten 3y ago

Appreciated the How is Babby Formed reference in chapter two:)

setheron 3y ago

How does one debug the Linux kernel ? I want to step through some portions. Even with UML it sounds arduous