8 ms·

One practice I don't think I've ever really seen properly discussed is exploratory debugging. Given a large codebase I'm not familiar with, I've found that this

by pdpi 8y ago

One practice I don't think I've ever really seen properly discussed is exploratory debugging. Given a large codebase I'm not familiar with, I've found that this workflow is an incredibly fast way to get familiar with the overall flow and structure of the code: - run the thing - grab an output string of some sort (either output proper or logs or whatever else) - grep for it - set a breakpoint where the string is generated - run again From there, you can explore the stack, and you start to get your bearings. How is this particular thing generated? Find the place in the stack where it first shows up, set a break point, start over and debug from there. Repeat until you have a sufficiently solid grasp of the codebase that you can just navigate the source tree searching for where things are.

Kagerjay 8y ago
Whenever I explore a library, I do the same, but when I think of CRUD functions I always look for these first in this general order: - Create methods - Delete methods - Update Methods - Read Methods Creating and deleting things are usually the easiest starting points. You always have an idea on how an app does this so its pretty easy to figure what things you need to do to trigger the breakpoint. Usually the create and delete methods are named something obvious so grepping for it is not terribly difficult If you work with JS libraries there is always some demo website, you can check for any event listeners on that element too, set breakpoints go from there.
- deleted 8y ago
  [deleted]
segmondy 8y ago
To each their own, I use to enjoy such. I absolutely hate it now. Given a codebase, I want to know the story. Why was the code written, what problem was being solved, why did they decide to solve it in a specific ways, WHY, WHY, WHY? I need to know the whys. I want the story, I want to know the domains. How is the software layered, can I explore one layer without going into other layers? I don't want to do so with a debugger. I want to do so with a document reader.
- dalbasal 8y ago
  The downside is that this assumes a "why" exists. Codebases are often evolved rather than designed, and just are the way they are without persistent reasons for things. A common reason would be "because it was this other way before." Imagine all the stuff that's probably in MS office, for which the reason is totally lost. There were reasons, but they're not really retrievable.
  - GrinningFool 8y ago
    > Codebases are often evolved rather than designed, and just are the way they are without persistent reasons for things. A common reason would be "because it was this other way before." Yes and no, I think. Codebases represent a collection of point-in-time decisions, and each one of those decisions had a reason. There's always a 'why' even if it's not captured. > Imagine all the stuff that's probably in MS office, for which the reason is totally lost. There were reasons, but they're not really retrievable. True - and that's Really Bad. That means you're stuck not able to improve or change certain things, because you don't know why they were done that way - and when you have a userbase the size of Office, you can't just say 'f it' and break things - at least not lightly, as a code-level decision.
    - throwawayjava 8y ago
      The worst "why?" answers often go like this: 1. Green field+0 years: This particular portion of the system was designed in this way as a cargo cult. 2. Green field+2 years: Because that portion of the system was designed in that way, we had to modify this other portion of the system to achieve X. 3. Green field+3 years: Because of the change in 2, and this new constraint Y, we had to make this other change in a third portion of the system. 5. Green field+(3 to N) years: repeat steps 2 and 3. 6. Green field+N+M years: Wow 5 (or previous 6) has created a mess and exposed a serious architectural flaw from step 1 or previous 6. All of the logic in the various iterations of 2+3 are refactored as best as possible. So, the "why" basically grounds out in "the refactoring we could afford to do in order to correct a mess that resulted from a cargo cult or previous refactoring of even older cargo cult". Is that "why" functionally helpful? Usually not for me -- in either case, I can't really assume a lot and I have to proceed with laborious "just watch what the code does" debugging/logging that I would've done without the "why?". But this is probably a ymmv situation.
specialist 8y ago
I do much the same. Alas, inferring flow of control (stepwise) is challenging with async operations. But even harder is inferring the data flows. So many abstractions, wrappers, transforms, slice & dice. Especially with these dynamic languages. So in desperation I'll use/write "tracing wrappers" over interfaces to record/log inputs and outputs. One example, I created code generating TraceGL (wrapper for OpenGL). Run my app, render a few frames, run the generated code, tweak til it works, then use that knowledge to fix my app. One recent example, code generating wrappers for S3, DynamoDB, Redis. So I can figure out exactly what data is going where. Then I can fix the code. Rob Pike has a quote about the primacy of data over code.
someguy1234567 8y ago
This process gets really fun when you don't have a debugger. Just source and program. Add log, recompile. Add log, recompile. oy.
- throwawayjava 8y ago
  I actually prefer logs to interactive debugging -- grepping through 20 different execution traces is much more pleasant than stepping through them, and I can discover everything instead of stepping through N times to fill out my mental image as new information becomes salient from previous debugging sessions. Also, I have never learned how to interactively debug anything parallel or especially distributed :( But I also try to write/architecture my code so that the big stuff always gets logged. In cases where the logging isn't as good as it should be, a few hundred lines of reflection/meta-programming can enable just enough aspect-oriented programming capabilities to enable this approach.
- AstralStorm 8y ago
  Additional fun when you are debugging something timing dependent like a livelock or a race condition.
pjc50 8y ago
This is an unreasonably effective tactic. It even works if you don't have any source code; it's usually possible to get the disassembler to tell you where the start address of a string is being loaded into a register, so just breakpoint that and you've got your reverse engineering starting point. Or just overwrite it with another string. "Thank you for playing Wing Commander". Wielding the low-level binary debugger well is also a good way to unnerve your coworkers and survive weird situations, like when you've got no tools because you've broken your tools with your tools ( https://www.usenix.org/system/files/1311_05-08_mickens.pdf https://www.usenix.org/system/files/1311_05-08_mickens.pdf ; content warning for hilarious hyperbolic rant)
- mjevans 8y ago
  Nice rant, good read, and I find my self wanting to know how they actually solved the problem.
wilsonnb2 8y ago
I currently work at a small development shop whose main product is a debugger. The process you describe is our standard way of navigating the code base - load the debugger up in the debugger and poke around. I'm not a fan of this workflow. I like to start learning about something from the top down and there is basically no way to do that with this method. It's great for getting into the nitty gritty details of the code to solve a problem, but it's terrible for learning how the different chunks of code fit together or how the code looks from the top. For our developers with many years of experience, this isn't a problem. For newcomers like me, it is. It probably doesn't help that this particular debugger is both written in and used for assembly language. If it was in a higher level language, I could probably get more of a top level view from the commentary, meaningful variable names longer than 8 characters, and the fact that 10 lines of python does a lot more than 10 lines of C which does a lot more than 10 lines of assembly.
stcredzero 8y ago
In VisualWorks Smalltalk, you could ask all the strings in the running system if they matched a pattern, even with regexes. Then you could ask the matching strings for all the objects that held runtime references to them, and quickly browse them.
elliotlarson 8y ago
I totally agree. One of the things that keeps me in Ruby land is the absolutely phenomenal debugging/repl environment provided by Pry and related gems. When I want to understand how something works, I drop a binding.pry, run the code and start digging through the stack with the Pry toolchain. I feel like this process is so important and it's the first thing I find myself missing when I work with other languages. (I suspect you can get similar debugger/repl functionality with Python's iPython, but I don't have a lot of experience with it.)
Ace17 8y ago
The way you describe "exploratory debugging" make it sound very close to binary reverse engineering: you put breakpoints/tracepoints at strategic places, run the program repeatedly, dump strategic memory addresses ... in an attempt to get a grasp at what the program is doing. This is what you do when you can't rely on file names, variable names, function names, types, and comments. A codebase that make this the fastest possible workflow probably has some important design issues.
acchow 8y ago
This is pretty similar to how I learn new systems.
hinkley 8y ago
You can tell people place no value on this by looking at how hard the call stack is to follow. Abstractions on abstractions so you can’t tell where you’re coming from or going to. Heck, even logging frameworks are this way. I wish we could sit down and rewrite some of these critical libraries for legibility. Especially as the number of libraries we have to interact with continues to grow day by day.
icos 8y ago
Given a large codebase I was not familiar with, I wrote https://vlasovstudio.com/runtime-flow/ https://vlasovstudio.com/runtime-flow/ tool. It shows and records executed code in real-time just as you interact with an application. (Works only for .NET Windows programs.)

by pdpi 8y ago

Kagerjay 8y ago

Whenever I explore a library, I do the same, but when I think of CRUD functions I always look for these first in this general order: - Create methods - Delete methods - Update Methods - Read Methods Creating and deleting things are usually the easiest starting points. You always have an idea on how an app does this so its pretty easy to figure what things you need to do to trigger the breakpoint. Usually the create and delete methods are named something obvious so grepping for it is not terribly difficult If you work with JS libraries there is always some demo website, you can check for any event listeners on that element too, set breakpoints go from there.

deleted 8y ago
[deleted]

segmondy 8y ago

To each their own, I use to enjoy such. I absolutely hate it now. Given a codebase, I want to know the story. Why was the code written, what problem was being solved, why did they decide to solve it in a specific ways, WHY, WHY, WHY? I need to know the whys. I want the story, I want to know the domains. How is the software layered, can I explore one layer without going into other layers? I don't want to do so with a debugger. I want to do so with a document reader.

dalbasal 8y ago
The downside is that this assumes a "why" exists. Codebases are often evolved rather than designed, and just are the way they are without persistent reasons for things. A common reason would be "because it was this other way before." Imagine all the stuff that's probably in MS office, for which the reason is totally lost. There were reasons, but they're not really retrievable.
- GrinningFool 8y ago
  > Codebases are often evolved rather than designed, and just are the way they are without persistent reasons for things. A common reason would be "because it was this other way before." Yes and no, I think. Codebases represent a collection of point-in-time decisions, and each one of those decisions had a reason. There's always a 'why' even if it's not captured. > Imagine all the stuff that's probably in MS office, for which the reason is totally lost. There were reasons, but they're not really retrievable. True - and that's Really Bad. That means you're stuck not able to improve or change certain things, because you don't know why they were done that way - and when you have a userbase the size of Office, you can't just say 'f it' and break things - at least not lightly, as a code-level decision.
  - throwawayjava 8y ago
    The worst "why?" answers often go like this: 1. Green field+0 years: This particular portion of the system was designed in this way as a cargo cult. 2. Green field+2 years: Because that portion of the system was designed in that way, we had to modify this other portion of the system to achieve X. 3. Green field+3 years: Because of the change in 2, and this new constraint Y, we had to make this other change in a third portion of the system. 5. Green field+(3 to N) years: repeat steps 2 and 3. 6. Green field+N+M years: Wow 5 (or previous 6) has created a mess and exposed a serious architectural flaw from step 1 or previous 6. All of the logic in the various iterations of 2+3 are refactored as best as possible. So, the "why" basically grounds out in "the refactoring we could afford to do in order to correct a mess that resulted from a cargo cult or previous refactoring of even older cargo cult". Is that "why" functionally helpful? Usually not for me -- in either case, I can't really assume a lot and I have to proceed with laborious "just watch what the code does" debugging/logging that I would've done without the "why?". But this is probably a ymmv situation.

specialist 8y ago

I do much the same. Alas, inferring flow of control (stepwise) is challenging with async operations. But even harder is inferring the data flows. So many abstractions, wrappers, transforms, slice & dice. Especially with these dynamic languages. So in desperation I'll use/write "tracing wrappers" over interfaces to record/log inputs and outputs. One example, I created code generating TraceGL (wrapper for OpenGL). Run my app, render a few frames, run the generated code, tweak til it works, then use that knowledge to fix my app. One recent example, code generating wrappers for S3, DynamoDB, Redis. So I can figure out exactly what data is going where. Then I can fix the code. Rob Pike has a quote about the primacy of data over code.

someguy1234567 8y ago

This process gets really fun when you don't have a debugger. Just source and program. Add log, recompile. Add log, recompile. oy.

throwawayjava 8y ago
I actually prefer logs to interactive debugging -- grepping through 20 different execution traces is much more pleasant than stepping through them, and I can discover everything instead of stepping through N times to fill out my mental image as new information becomes salient from previous debugging sessions. Also, I have never learned how to interactively debug anything parallel or especially distributed :( But I also try to write/architecture my code so that the big stuff always gets logged. In cases where the logging isn't as good as it should be, a few hundred lines of reflection/meta-programming can enable just enough aspect-oriented programming capabilities to enable this approach.
AstralStorm 8y ago
Additional fun when you are debugging something timing dependent like a livelock or a race condition.

pjc50 8y ago

This is an unreasonably effective tactic. It even works if you don't have any source code; it's usually possible to get the disassembler to tell you where the start address of a string is being loaded into a register, so just breakpoint that and you've got your reverse engineering starting point. Or just overwrite it with another string. "Thank you for playing Wing Commander". Wielding the low-level binary debugger well is also a good way to unnerve your coworkers and survive weird situations, like when you've got no tools because you've broken your tools with your tools ( https://www.usenix.org/system/files/1311_05-08_mickens.pdf https://www.usenix.org/system/files/1311_05-08_mickens.pdf ; content warning for hilarious hyperbolic rant)

mjevans 8y ago
Nice rant, good read, and I find my self wanting to know how they actually solved the problem.

wilsonnb2 8y ago

I currently work at a small development shop whose main product is a debugger. The process you describe is our standard way of navigating the code base - load the debugger up in the debugger and poke around. I'm not a fan of this workflow. I like to start learning about something from the top down and there is basically no way to do that with this method. It's great for getting into the nitty gritty details of the code to solve a problem, but it's terrible for learning how the different chunks of code fit together or how the code looks from the top. For our developers with many years of experience, this isn't a problem. For newcomers like me, it is. It probably doesn't help that this particular debugger is both written in and used for assembly language. If it was in a higher level language, I could probably get more of a top level view from the commentary, meaningful variable names longer than 8 characters, and the fact that 10 lines of python does a lot more than 10 lines of C which does a lot more than 10 lines of assembly.

stcredzero 8y ago

In VisualWorks Smalltalk, you could ask all the strings in the running system if they matched a pattern, even with regexes. Then you could ask the matching strings for all the objects that held runtime references to them, and quickly browse them.

elliotlarson 8y ago

I totally agree. One of the things that keeps me in Ruby land is the absolutely phenomenal debugging/repl environment provided by Pry and related gems. When I want to understand how something works, I drop a binding.pry, run the code and start digging through the stack with the Pry toolchain. I feel like this process is so important and it's the first thing I find myself missing when I work with other languages. (I suspect you can get similar debugger/repl functionality with Python's iPython, but I don't have a lot of experience with it.)

Ace17 8y ago

The way you describe "exploratory debugging" make it sound very close to binary reverse engineering: you put breakpoints/tracepoints at strategic places, run the program repeatedly, dump strategic memory addresses ... in an attempt to get a grasp at what the program is doing. This is what you do when you can't rely on file names, variable names, function names, types, and comments. A codebase that make this the fastest possible workflow probably has some important design issues.

acchow 8y ago

This is pretty similar to how I learn new systems.

hinkley 8y ago

You can tell people place no value on this by looking at how hard the call stack is to follow. Abstractions on abstractions so you can’t tell where you’re coming from or going to. Heck, even logging frameworks are this way. I wish we could sit down and rewrite some of these critical libraries for legibility. Especially as the number of libraries we have to interact with continues to grow day by day.

icos 8y ago

Given a large codebase I was not familiar with, I wrote https://vlasovstudio.com/runtime-flow/ https://vlasovstudio.com/runtime-flow/ tool. It shows and records executed code in real-time just as you interact with an application. (Works only for .NET Windows programs.)