11 ms·

The Debugging Mindset

246 points by jodooshi 10y ago

unexistance 10y ago
"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?" Timeless If you have to use 'clever' hacks, the least you can do is justify it in the comment
- AstralStorm 10y ago
  Fortunate that a good programmer will get more clever over time. Also will know when to write boring and or testable code. Testable code is also debuggable more easily.
- blauditore 10y ago
  And verify it with good tests. It sometimes happens that a certain piece of code is inherently complex, due to a complex underlying algorithm. But often, test cases are still easy to define, so it makes sense to heavily make use of them. For example, sorting algorithms may be somewhat complicated in implementation, but it's trivial to come up with some sample before/after lists.
cubano 10y ago
I have always used a step-debugger for debugging and personally find the process somewhat thrilling, especially when debugging other peoples code and learning all the tricks and techniques that they are using to get the jobs done. I don't know how much time and effort I have saved over the years using tricks I gleaned stepping through code written by people much smarter than I. There is simply nothing that comes close to learning and debugging code line-by-line, examining variables and taking branches in real time. In fact, how people can debug large existing codebases without a debugger is beyond me. Of course, nowadays in modern webdev it seems that most developers no longer use step-debuggers and, in general, want to rewrite the app in whatever stack will look best on their resume and/or sound impressive in the pub or gym afterhours. I long ago gave up poo-pooing the importance of social signalling within the young-ish developer community, something that really did not exist when I was 20-something as programmers had few choices about the stack. I really love the jist of the OP, and agree that the misapplication of the programs mental model is the underlying cause of almost all bugs. This mental model is what I'm always trying to load fully into my brain as, IMO, true productivity occurs when I have it fully loaded and I am confident that I am covering all the edge conditions. Debugging must not be an afterthought in educating; industry must stop insisting that bugs be interpreted as failures of individual programmers ... Funny...I've really never separated debugging from developing as they always go hand in hand, and certainly have never thought that bugs signify "failure". In fact, to me it represents progress...
- henrik_w 10y ago
  I agree - debugging is an integral part of programming. I don't mind debugging, and often bugs are good for you: - solving a bug teaches you something. You either learn something new about how to write good code, about the product or about the domain. - debugging problems teaches you how to write code that is easy/easier to debug. You see what information you wish you have had, how to be able to track what is happening in the program etc. - solving problems and mysteries (how can this even happen) is often fun in itself (like solving sudoku or crosswords or reading murder mysteries). - when you resolve a bug, both you and your customer are happy. It's a bit of a paradox. If there wasn’t a bug in the first place, there wouldn’t be a need to fix it, so why should they be happy? However, my experience is that they are happy to receive a bug-fix, especially if it is solved quickly More detailed argument here: https://henrikwarne.com/2012/10/21/4-reasons-why-bugs-are-good-for-you/ https://henrikwarne.com/2012/10/21/4-reasons-why-bugs-are-go...
  - mrb 10y ago
    "debugging is an integral part of programming" Can't remember where I read this: programming is 20% writing bugs, and 80% fixing them.
- aidos 10y ago
  I recently fixed a bug in fontforge to do with extracting fonts when parsing pdfs. It took me a whole day because I didn't know what I was doing (haven't worked with C since Uni over 15 years ago). Because I didn't think to look into using a debugger, I did it very much the hard way by adding trace statements all over the place to slowly narrow down the code path. I knew I was looking for something like that looked like an iteration typo, but it was a large unfamiliar codebase so it tok a long time. Got there in the end! Probably 2 days of work start to finish for a 1 character fix. Classic. I will definitely be starting by re-learning to use a step debugger in C next time I'm in the same position. https://github.com/fontforge/fontforge/pull/3050/commits/e4b746c71eaa8b747220cf93fc380f9dbd0a835d https://github.com/fontforge/fontforge/pull/3050/commits/e4b...
- NLips 10y ago
  "In fact, how people can debug large existing codebases without a debugger is beyond me." I currently work on a product with a 15-20 Mloc codebase (mostly C). The development facets which give the closest parallel to a line-by-line debugger are: Components in the system communicate with each other by sending signals through the framework. All such signals are traceable in all builds, and can easily be decoded. These narrow down which components is doing something wrong, and what exactly it's black-box behaviour is (i.e. when I see bug B, it's because component C is giving output O after receiving input I). Lots of trace. All branches are traced in debug builds, unless there's a serious perf reason not to. Many interesting variables are traced too. Extensive UT and FV, using powerful (rubbish in other ways) frameworks. Any real-world bug should be reproducible in automated tests, which you can build and run with debug-level trace. It's not always fast, but with these you can always find the problem line / section with typical code bugs. Obviously some types of bug (e.g. perf) often don't come down to 'a line of code' or 'a bad interaction', so aren't necessarily resolved this way, but that's the same with line-by-line debuggers, in my experience.
jgrahamc 10y ago
“By June 1949, people had begun to realize that it was not so easy to get a program right as had at one time appeared. It was on one of my journeys between the EDSAC room and the punching equipment that the realization came over me with full force that a good part of the remainder of my life was going to be spent in finding errors in my own programs.” https://research.swtch.com/discover-debug https://research.swtch.com/discover-debug
filleokus 10y ago
I think it's fascinating to see how my friends at university (who previously had no programming experience) have started to "get" debugging and the benefits of using something like a step-debugger. It took several programming courses before they started to actually debug the code instead of just reading it and trying to figure out what happens. I mean, reading the code is also an integral part of debugging, but often times they said things like "fooBar _should_ be true, so we _should_ enter this conditional". Then it of course turned out that fooBar was in fact not true because of some logic bug or something they didn't even consider.
- wapz 10y ago
  The problem I had at the university was that our code was so simple relative to large codebases that a debugger was generally not necessary for me. We learned how to use it and I found it to be a burden because I could find the bug "much quicker" either with 1-2 print statements or just by reading it. After working with much bigger problems I realized the importance of debuggers and really wished they were stressed even more than they were in my university days.
- hamstercat 10y ago
  I had the same experience at University. We didn't learn how to use a debugger and rather relied on logging data to see the flow of the program. That, plus the fact that our code was a big mess since we were just starting didn't make it easy to find the more obscure bugs. I was really impressed the first time my friend fired up Netbeans and showed me that how he could find the value of the variables at runtime, at any point in time of the execution. Today I find it unacceptable when I'm in a situation where I can't debug a program I'm working on, it just seems like a waste of time.
sharpercoder 10y ago
>Are the arguments to memcpy in the order of source, destination, length; or are they destination, source, length? Is strstr haystack, needle; or is it needle, haystack? Many individuals develop a habit of consulting references (such as system manuals) as soon as the question comes up. I've always found this particular problem solveable by allowing programming languages having syntax in the form of public void CopyFrom(int address)To(int address)OfSize(long size) { // impl } instead of Copy(int fromAddress, int toAddress, long size) Are there good reasons why languages haven't adopted this syntactic nicety?
- seanwilson 10y ago
  Surely having your IDE bring up the argument names during autocomplete solves this?
  - fit2rule 10y ago
    Dependence-on-IDE is a liability. Ability to find and read the docs: an asset.
    - seanwilson 10y ago
      > Dependence-on-IDE is a liability. Why? Use the best tool for the job. > Ability to find and read the docs: an asset. Read the docs + use a good IDE.
    - edejong 10y ago
      Yes, I don't see this point either. A 'man 3 stpncpy' or an alt-space seems rather similar to me. Actually, with the 'man 3 stpncpy' we cannot be sure the results will align with the target platform. Everyone uses IDEs, whether it is Emacs, Vim (with plugins), IntelliJ, Eclipse, Atom, Xcode doesn't matter. Yes, I also use whiteboards, A4's, a notepad, mindmaps (Freemind), post-its and more. The creative parts of SE, such as architecture and design, do not blossom behind a computer.
      - seanwilson 10y ago
nostrademons 10y ago
The scientific-method-like "general approach to debugging" that the author describes in the last third of the paper works, but it's pretty inefficient. The number of possible hypotheses that may cause an observed bug is very large; if you test each individually, you may have to go through many permutations, and you're not systematically gathering data that may help you narrow the search space. Rather, I've found "divide and conquer" debugging to be much more effective. You should know (based on your domain knowledge of the program) the rough sequence of operations between the input and output of your code; if you don't, try single-stepping through it or adding log statements. Pick a point approximately halfway across that computation, and inspect (via debugger, println, or unit tests) the state of the program at that point. Does it match your mental model? If so, the bug is between there and the output; repeat the process on the latter half of the code. If not, the bug is between the input and your intermediate point; repeat the process on this subset. As an added benefit, you can often find and fix other, unrelated bugs that may not be symptomatic yet but are silently waiting to trigger strange results. This process is O(log N) in the size of the code path, while the "formulate hypothesis and test it" approach is potentially combinatoric if the code interacts with itself.
- lmitchell 10y ago
  This is a surprisingly accurate description of the general sort of debugging technique I've gotten better at as I get better at programming. I find that logically it tends to make more sense to me to start with a high-level function, and step over its component function calls/loops one by one, inspecting the program state and diving into any that seem to be producing strange results. But overall, I definitely just find it's a lot more effective to start from a broad 'does everything look okay at this point?' than from a narrow 'I think the problem is this'.
  - Swizec 10y ago
    > 'does everything look okay at this point?' than from a narrow 'I think the problem is this'. Even in a small real world codebase with just a few tens of thousands of lines and a couple hundred possible user interactions, you need the intuition for "I think the problem is this" or you're gonna be there for the rest of the year stepping through the code. Once you've scoped the problem to a potential few tens of functions, then you can spelunk through it to find the exact issue.
theaeolist 10y ago
Debugging an API, which may have obscure corner cases or usability quirks seems appropriate to me. "Debugging" an algorithm which has inherent logical flaws by attempting to patch it up with case-base reasoning and exceptions (logical, not runtime) is a road to disasters. I have seen too many devs trying to hill-climb their way out of a logical hole and the result is invariably a mess of a program which is still wrong. Also, in case of concurrent and parallel code debugging is basically useless since realistic timing conditions are not reproduced. I personally prefer logging, which gives a faster and more targeted trace of program behaviour than step debugging.
sleepychu 10y ago
My favorite debugging trick (when faced with an 'impossible' bug) is to state my assumptions, it's usually clear when they're all laid out which ones are conflicting.
- mrSugar 10y ago
  Sounds like a variation on Ayn Rand's "contradictions do not exist, if you think you see one, check your premises, some of them will be wrong".
  - eriknstr 10y ago
    Is that quote intentionally saying that you have contradicting premises, and by extension is the statement in that sentence that contradictions do not exist contradicted by the later part of the sentence, or is there something about the sentence that I'm not understanding?
    - mysterydip 10y ago
      It reads like that at first. What it's saying is for example: Assume x is 5. Assume function TimesFive multiplies things by 5. When we do TimesFive(x) and my answer is 50 instead of the 25. I check the TimesFive function and it's just "print x*5". But how is that possible? 5x5 is 25! The problem in this case is my assumption that x would be 5 was wrong. I've had this issue before in my code (though not as trivial an example of course), but it's always a good reminder of something to check when troublshooting.
  - dhodell 10y ago
    Which sounds like a variation on various Socratic learning techniques.
- jspash 10y ago
  Sounds like Rubber Duck Debugging. A simple, yet effective technique. http://wiki.c2.com/?RubberDucking http://wiki.c2.com/?RubberDucking https://blog.codinghorror.com/rubber-duck-problem-solving/ https://blog.codinghorror.com/rubber-duck-problem-solving/ Having just read the recent article on HN about Japanese workers "pointing at things" got me thinking that the two actions probably stimulate a similar area of the brain. Sometimes we just need to get "out of our head" to make a breakthrough in the problem we're trying to solve.
_nalply 10y ago
My secret sauce to debugging is formulating hypotheses about what went wrong then instrument the software to validate the hypotheses. By instrumenting I am talking about manipulating running software to give me information. Sometimes the manipulating step is made unnecessarily hard, and the trick is to force the hand. I see myself an inquisitor of software and use programmatic torture to get what I want to know. That's the central attitude to debugging. The devil is in the details, there are many possibilities to get information: using a runtime debugger, insert output statements, read the logs, use LD_PRELOAD, use strace, and so on.
mattmanser 10y ago
I personally found this to be a poor article on the subject. There's very little actual content about debugging, he spent more time talking about theories of intelligence than debugging. I've recently been thinking about this, thinking about making a course. It's my personal experience that his approach is flawed. Fundamentally, step one is not develop a theory. That is the worst thing you can do, it can waste a lot of time. I still sometimes guess what the problem is, but all I really use that for today is where to look and that's sometimes a waste of my time as I skipped step 1. Step 1 is recreate it. No theorizing, no thinking, no faffing around. Can you recreate the bug exactly? If you can't, find out why. Talk to the bug reporter, find out what they were doing, this is a people skill you need to learn as a programmer. Maybe you need access to the live data to see the state an object is in. There's only a tiny percentage of bugs (usually race conditions) that can't easily be recreated. If you can't recreate it, that's when you turn to logging to try and catch the conditions to recreate it, but logging is the last thing you should try and I rarely ever have to add it (we're talking 1% of bugs). I often see junior programmers littering code with log statements when trying to debug, I feel this is a bad method, but can understand the temptation, especially when they don't really understand what the code is doing (an understandable position I was in in my earlier days). Step 2 is isolate it. If the code is simple and it's obviously one method, no further work needed. If the code is complex, you need to really isolate the exact problem. Say you've got the wrong value displaying for a basket total, where is that coming from? Is it not updating, or is the total function the problem? Isolate the exact step in the process that's going wrong. Sometimes it's hard to isolate and logging is needed again, but same advice as before, logging is a rare necessity. Step 3 is make it easy to run a recreation. If you can run it in 10 secs or less, that's great. If it's one method of a complex web call, make a test endpoint that only calls that method that's failing and exactly recreates the conditions. Then you can run it quickly without all the cruft. In an extreme case, this can be taking the code completely out of the context and putting it in a tiny standalone app. Step 4 is read + understand the code. Is there an obvious logic bug? Don't jump to conclusions. Still don't get what's going on? Step through the code, breakpoints. Understand what's going wrong. Breakpoints and stepping is fine, inspecting variables, etc. Step 5 is fix it. This should be fairly simple now. This is also the time to think about the future. Is this part of the code a constant headache? Is it buggy because it's over-complicated? It might need a bit of a refactor to make sure this doesn't happen again. This is a judgement call, unnecessary refactors are expensive, might even introduce new bugs, but over-complicated code will lead to more bugs. It not being written in your favourite style is generally not a legitimate reason to refactor. Step 6 is testing the fix, recreating the conditions from step 1. Don't rely on QA if you have them. Ensure you've done your job. (Step 7 get rid of your logging statements + test it again! You just changed the code) Now you can skip steps, be far less formal about it. Sometimes it's obvious. Sometimes it's hard and you may need to iterate over 4/5/6 a few times. But never, ever skip steps 1 + 6. Recreate it. Test your fix. Because if you skip those steps, and you "fix" it, you don't actually know it's fixed.
jerianasmith 10y ago
Investigating is essentially a space particular term for critical thinking. Bugs are depicted as word problems.Since mental models are approximations, they are in some cases inaccurate, prompting unusual conduct in programming when it is created on top of flawed suspicions.
rdtsc 10y ago
To make debugging easier you have to pick tools (languages, IDEs, methodologies) that make it easier. In other words you optimize for debugging. For example, I find Erlang code very easy to debug. There are few reasons for that: * Built-in runtime tracing is easy. Can remsh into a VM node and trace any function at runtime. * Functional style with immutable data and variables make it easy to figure what is changing in the code. Because of immutability the state is always updated explicitly. That makes it much easier to understand what is happening for me that figuring it out than having a class inheritance hierarchy with methods overrides and such in an OO language * Code hot-patching makes it easier to test and iterate to zoom on in a bug. With care it can be done in production. You see strange failure you never saw before and which you had an extra log statement in there? That's easily done. * Language is simple and self-consistent. That means there is less chance someone used some obscure new feature I haven't heard about. This is a bit like C vs C++. C is very simple at the language level. You can still have hairy code with triple pointers, funky dispatch tables with Duff's Device thrown in there but at least you can read through it. Looking at C++ code I sometimes couldn't even parse what was happening because of a language feature (so they overrode the operator some place, and using a templates, ...). Or someone created a macro and then everything is more interesting all of a sudden. * Process heap isolation. When something happens with one process, I know for a fact that it hasn't scribbled on memory of other processes. I can restart that process if needed and test my hypothesis of what I think is wrong, often without affecting the rest of the system (even while it is running in production). * Sane concurrency primitives: Having the ability to create hundreds of thousands of processes with isolated memory often mean a smaller impedance mismatch with your business code so you have less total code to write. The less code your write, the less code you have to debug. Anyway the point being, debugging should be a trade-off in picking your tools. People don't think about it much and focus on other things first. But it will be something that over time will make a tremendous difference. > Virtual machine-based languages, interpreted languages, and languages with runtime environments encourage users to view the execution environment as a black box. The goal here is to make programming easier by reducing the scope of the mental model the programmer must maintain. When bugs occur in these execution environments, you're left with a complete gap in understanding. It can be the opposite as well. Having a VM with good tracing and debugging facilities is very useful. Often you can do things wouldn't be able to with just plain compiled code.
noyeda 10y ago
Nice article. Thought I'd note something: > Software developers spend 35-50 percent of their time validating and debugging software.1 This is a strong claim, so I looked at the cited source, given the unfortunate experience of having seen many strong claims backed by weak underlying data. > The research phase will predominantly comprise of short interviews with approximately 10 to 12 organisations that compile code on the LINUX operating environment. These organisations will also fill in the Cambridge Venture Project (CVP) survey to help quantify how much time they spend debugging with and without RDBs. Broader research in the form of the CVP survey with potential users of reversible debuggers will be conducted to gain insights on how much time is currently spent debugging. Further down: >49.9% Programming time spent debugging > According to 54 questionnaire responses to the CVP survey and 11 interviews, based on the question ‘Of the time spent programming, what percentage of your time is spent on the following: (1)fixing bugs (2)making code work (3)designing code (4) writing code.’ (1) and (2) then were grouped as debugging. While this might be indicative of a widespread phenomena, it doesn't seem to be enough evidence to support the factual and strong claim of the first sentence in the ACM article. There certainly isn't enough information in the cited source alone to decide (and given that, is this really sufficient as a source?). Perhaps I missed further evidence? That being said, I won't throw the baby out with the bathwater. Debugging is clearly a time sink in industry, regardless of what the actual percentage is. I enjoyed this piece overall. I was previously unfamiliar with some of these terms -- specifically incremental vs. entity mindsets -- and it shed some light on work issues I've seen (coincidentally in the debugging technology space). Being someone with an incremental mindset and having worked for someone with an unjustified entity mindset, I found the environment suffered from almost all the problems (and more) you described associated with the latter mindset. Granted, this is anecdotal. I quite strongly agree that improving the process and mindset behind problem-solving results in a significant improvement to specific skills like debugging software. It's refreshing to see an article like this instead of the many others hyperfocused on specific, less-abstract techniques (and widely-applicable abstract techniques/processes are the most effective use of one's learning time).
- dhodell 10y ago
  Thank you for the feedback. I chose (for better or worse) to cite the most recent article as opposed to maybe the least controversial one. Other papers I've read have suggested similar ranges; nobody presents a stddev so it's hard to compare. And how you classify "debugging-related tasks" is kind of subjective, as you correctly point out. So linking other papers might or might not be useful anyway. In "A framework and methodology for studying the causes of software errors in programming systems" (Ko & Meyers 2005), they cite a NIST publication as saying 70-80% of time is spent debugging. I parroted this off in a talk once, and then I looked at the same publication ("The Economic Impact of Inadequate Infrastructure for Software Testing" RTI 2002) and the only 80% I found was time spent in testing software in the early days of software engineering. In the 1990s, it finds coding and unit testing to comprise about 30% of the time. This is as far as I can tell, at least similar to the (1) and (2) in the CVP survey. But again, indirect comparisons, who knows? In tables 6-4 and 7-6, the RTI study finds different numbers for time spent on bugs, and different frequencies of where bugs end up being discovered. Whatever the rate, it is clear that post-production discovery bugs are the hardest to fix, and I think this is because you only ship code when you have high confidence in its correctness. I think this is a hard measurement to make. It's hard to find participants, hard to guarantee they have the same idea of what "debugging means", hard to quantify. So thank you for calling me out on this. But I think other studies have evidence that this isn't too far off from a correct measurement. For example, Gugerty and Olson in 1986 did some novice vs expert study at debugging LOGO programs. (I don't like the Dreyfus model for classifying debugging, but whatever.) They found that novices took about 18 minutes to fix bugs, testing an average of 3.6 hypotheses per program, with the first hypothesis being correct only 21% of the time, and re-executing code every ~3 minutes. Experts took 7 minutes, testing an average of 1.6 hypotheses per program, with the first hypothesis being correct 56% of the time, re-executing code every 2 minutes. So guesses are only right maybe half the time if you're really good. Basically any novice vs expert study compares how much time people spend on the problem (they're trying to reduce debugging time), but it's hard to extrapolate this into percentage of development time. For example, some people spend 100% of their time debugging because they're on QA or sustaining engineering teams. To be quite honest, there _fundamentally is not_ enough research into the practice and pedagogy of debugging. I have about 60 papers in my "Debugging Papers" folder that I've read over the past year, and basically everything up until the Dweck-inspired research in 2008 isn't great. There are only 9 papers in all of computer science that cite Dweck that I've found as of maybe 6 months ago (I'm not a researcher and I don't look that often). From memory, two of them were recommending further research based on her work, maybe five of them were from people in CS departments helping Dweck create games and programs to perform her research, and then only two were actually applying her work to debugging. I would agree with you if you said this needs more research, and that the claim isn't strongly supported by the provided evidence. I greatly appreciate your feedback.
sn9 10y ago
Udacity actually has a course on systematic debugging taught by the creator of DDD. The course covers the importance of the scientific method and hypothesis testing, divide and conquer approaches, etc.
edejong 10y ago
Of course, Dijkstra had his view on it. I try to adhere to his position and prefer proof by type-checking over proof by single example. "A common approach to get a program correct is called "debugging" and when the most patent bugs have been found and removed one tries to raise the confidence level further by subjecting the program to numerous test cases. From the failures around us we can derive ample evidence that this approach is inadequate. To remedy the situation it has been suggested that what we really need are "automatic test case generators" by which the pieces of program to be validated can be exercised still more extensively. But will this really help? I don't think so." [1] Linus (used) to have a similar position [2] Debugging hides the underlying architectural problem: to either communicate or abstract the algorithm in such a way that it can be understood by the programmer. If something needs debugging, not the bug needs to be fixed, but the architecture to understand the bug before debugging. [1] Edsger Wybe Dijkstra - EWD303. https://www.cs.utexas.edu/users/EWD/transcriptions/EWD03xx/EWD303.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD03xx/E... [2] http://lwn.net/2000/0914/a/lt-debugger.php3 http://lwn.net/2000/0914/a/lt-debugger.php3
- dhodell 10y ago
  Thanks for this. This seems to be basically what I'm saying through most of the article. Bugs occur because of a gap in a mental model. To fix the gap, we need more understanding. I'm not sure I would say that's necessarily a design or architecture flaw in the software. Unfortunately, proving things correct in most popular languages is a Very Hard Problem. I do address that to some extent in the article, though. In particular, the sorts of languages that end up being easy to prove correct tend to be difficult for people to learn. Therefore, people are likely to either learn something easier, or they are likely to write code that is "correct" or "safe" but still has logic errors. I think that there are other many valid reasons to write software, and not all of them require correctness to the extent Dijkstra would like. But I think that's a pipe dream anyway. For one, most software is not written in formally verifiable languages, or formally verifiable dialects of popular languages. Porting all software would be impossible. Second, who defines "correct"? What about bug-compatible software, where the definition of correct is imitating something else that is known to be incorrect? What about ambiguity in specifications like JSON, or undefined behavior in C (some of which can only be detected at runtime, because we don't actually have infinite tape in our Turing machines)? What happens when there's a bug in the runtime of your formally verifiable language? What happens when there's a hardware bug? I'm not saying these are necessarily common, but regardless of your software environment, you're running on something else that can have the same bugs. People make mistakes, even when proving things. Math is great, but people make mistakes, and our tools are still bounded by the laws of physics. No matter which way I look at this thing, I keep coming back to teaching and understanding as the only generalizable and approachable ways to address the problems bugs. (Whether that approach is taken by proving correctness of some implementation, or post-hoc with testing and tools.)

246 points by jodooshi 10y ago

"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?" Timeless If you have to use 'clever' hacks, the least you can do is justify it in the comment

AstralStorm 10y ago
Fortunate that a good programmer will get more clever over time. Also will know when to write boring and or testable code. Testable code is also debuggable more easily.
blauditore 10y ago
And verify it with good tests. It sometimes happens that a certain piece of code is inherently complex, due to a complex underlying algorithm. But often, test cases are still easy to define, so it makes sense to heavily make use of them. For example, sorting algorithms may be somewhat complicated in implementation, but it's trivial to come up with some sample before/after lists.

cubano 10y ago

I have always used a step-debugger for debugging and personally find the process somewhat thrilling, especially when debugging other peoples code and learning all the tricks and techniques that they are using to get the jobs done. I don't know how much time and effort I have saved over the years using tricks I gleaned stepping through code written by people much smarter than I. There is simply nothing that comes close to learning and debugging code line-by-line, examining variables and taking branches in real time. In fact, how people can debug large existing codebases without a debugger is beyond me. Of course, nowadays in modern webdev it seems that most developers no longer use step-debuggers and, in general, want to rewrite the app in whatever stack will look best on their resume and/or sound impressive in the pub or gym afterhours. I long ago gave up poo-pooing the importance of social signalling within the young-ish developer community, something that really did not exist when I was 20-something as programmers had few choices about the stack. I really love the jist of the OP, and agree that the misapplication of the programs mental model is the underlying cause of almost all bugs. This mental model is what I'm always trying to load fully into my brain as, IMO, true productivity occurs when I have it fully loaded and I am confident that I am covering all the edge conditions. Debugging must not be an afterthought in educating; industry must stop insisting that bugs be interpreted as failures of individual programmers ... Funny...I've really never separated debugging from developing as they always go hand in hand, and certainly have never thought that bugs signify "failure". In fact, to me it represents progress...

henrik_w 10y ago
I agree - debugging is an integral part of programming. I don't mind debugging, and often bugs are good for you: - solving a bug teaches you something. You either learn something new about how to write good code, about the product or about the domain. - debugging problems teaches you how to write code that is easy/easier to debug. You see what information you wish you have had, how to be able to track what is happening in the program etc. - solving problems and mysteries (how can this even happen) is often fun in itself (like solving sudoku or crosswords or reading murder mysteries). - when you resolve a bug, both you and your customer are happy. It's a bit of a paradox. If there wasn’t a bug in the first place, there wouldn’t be a need to fix it, so why should they be happy? However, my experience is that they are happy to receive a bug-fix, especially if it is solved quickly More detailed argument here: https://henrikwarne.com/2012/10/21/4-reasons-why-bugs-are-good-for-you/ https://henrikwarne.com/2012/10/21/4-reasons-why-bugs-are-go...
- mrb 10y ago
  "debugging is an integral part of programming" Can't remember where I read this: programming is 20% writing bugs, and 80% fixing them.
aidos 10y ago
I recently fixed a bug in fontforge to do with extracting fonts when parsing pdfs. It took me a whole day because I didn't know what I was doing (haven't worked with C since Uni over 15 years ago). Because I didn't think to look into using a debugger, I did it very much the hard way by adding trace statements all over the place to slowly narrow down the code path. I knew I was looking for something like that looked like an iteration typo, but it was a large unfamiliar codebase so it tok a long time. Got there in the end! Probably 2 days of work start to finish for a 1 character fix. Classic. I will definitely be starting by re-learning to use a step debugger in C next time I'm in the same position. https://github.com/fontforge/fontforge/pull/3050/commits/e4b746c71eaa8b747220cf93fc380f9dbd0a835d https://github.com/fontforge/fontforge/pull/3050/commits/e4b...
NLips 10y ago
"In fact, how people can debug large existing codebases without a debugger is beyond me." I currently work on a product with a 15-20 Mloc codebase (mostly C). The development facets which give the closest parallel to a line-by-line debugger are: Components in the system communicate with each other by sending signals through the framework. All such signals are traceable in all builds, and can easily be decoded. These narrow down which components is doing something wrong, and what exactly it's black-box behaviour is (i.e. when I see bug B, it's because component C is giving output O after receiving input I). Lots of trace. All branches are traced in debug builds, unless there's a serious perf reason not to. Many interesting variables are traced too. Extensive UT and FV, using powerful (rubbish in other ways) frameworks. Any real-world bug should be reproducible in automated tests, which you can build and run with debug-level trace. It's not always fast, but with these you can always find the problem line / section with typical code bugs. Obviously some types of bug (e.g. perf) often don't come down to 'a line of code' or 'a bad interaction', so aren't necessarily resolved this way, but that's the same with line-by-line debuggers, in my experience.

jgrahamc 10y ago

“By June 1949, people had begun to realize that it was not so easy to get a program right as had at one time appeared. It was on one of my journeys between the EDSAC room and the punching equipment that the realization came over me with full force that a good part of the remainder of my life was going to be spent in finding errors in my own programs.” https://research.swtch.com/discover-debug https://research.swtch.com/discover-debug

filleokus 10y ago

I think it's fascinating to see how my friends at university (who previously had no programming experience) have started to "get" debugging and the benefits of using something like a step-debugger. It took several programming courses before they started to actually debug the code instead of just reading it and trying to figure out what happens. I mean, reading the code is also an integral part of debugging, but often times they said things like "fooBar _should_ be true, so we _should_ enter this conditional". Then it of course turned out that fooBar was in fact not true because of some logic bug or something they didn't even consider.

wapz 10y ago
The problem I had at the university was that our code was so simple relative to large codebases that a debugger was generally not necessary for me. We learned how to use it and I found it to be a burden because I could find the bug "much quicker" either with 1-2 print statements or just by reading it. After working with much bigger problems I realized the importance of debuggers and really wished they were stressed even more than they were in my university days.
hamstercat 10y ago
I had the same experience at University. We didn't learn how to use a debugger and rather relied on logging data to see the flow of the program. That, plus the fact that our code was a big mess since we were just starting didn't make it easy to find the more obscure bugs. I was really impressed the first time my friend fired up Netbeans and showed me that how he could find the value of the variables at runtime, at any point in time of the execution. Today I find it unacceptable when I'm in a situation where I can't debug a program I'm working on, it just seems like a waste of time.

sharpercoder 10y ago

>Are the arguments to memcpy in the order of source, destination, length; or are they destination, source, length? Is strstr haystack, needle; or is it needle, haystack? Many individuals develop a habit of consulting references (such as system manuals) as soon as the question comes up. I've always found this particular problem solveable by allowing programming languages having syntax in the form of public void CopyFrom(int address)To(int address)OfSize(long size) { // impl } instead of Copy(int fromAddress, int toAddress, long size) Are there good reasons why languages haven't adopted this syntactic nicety?

seanwilson 10y ago
Surely having your IDE bring up the argument names during autocomplete solves this?
- fit2rule 10y ago
  Dependence-on-IDE is a liability. Ability to find and read the docs: an asset.
  - seanwilson 10y ago
    > Dependence-on-IDE is a liability. Why? Use the best tool for the job. > Ability to find and read the docs: an asset. Read the docs + use a good IDE.
  - edejong 10y ago
    Yes, I don't see this point either. A 'man 3 stpncpy' or an alt-space seems rather similar to me. Actually, with the 'man 3 stpncpy' we cannot be sure the results will align with the target platform. Everyone uses IDEs, whether it is Emacs, Vim (with plugins), IntelliJ, Eclipse, Atom, Xcode doesn't matter. Yes, I also use whiteboards, A4's, a notepad, mindmaps (Freemind), post-its and more. The creative parts of SE, such as architecture and design, do not blossom behind a computer.
    - seanwilson 10y ago

nostrademons 10y ago

The scientific-method-like "general approach to debugging" that the author describes in the last third of the paper works, but it's pretty inefficient. The number of possible hypotheses that may cause an observed bug is very large; if you test each individually, you may have to go through many permutations, and you're not systematically gathering data that may help you narrow the search space. Rather, I've found "divide and conquer" debugging to be much more effective. You should know (based on your domain knowledge of the program) the rough sequence of operations between the input and output of your code; if you don't, try single-stepping through it or adding log statements. Pick a point approximately halfway across that computation, and inspect (via debugger, println, or unit tests) the state of the program at that point. Does it match your mental model? If so, the bug is between there and the output; repeat the process on the latter half of the code. If not, the bug is between the input and your intermediate point; repeat the process on this subset. As an added benefit, you can often find and fix other, unrelated bugs that may not be symptomatic yet but are silently waiting to trigger strange results. This process is O(log N) in the size of the code path, while the "formulate hypothesis and test it" approach is potentially combinatoric if the code interacts with itself.

lmitchell 10y ago
This is a surprisingly accurate description of the general sort of debugging technique I've gotten better at as I get better at programming. I find that logically it tends to make more sense to me to start with a high-level function, and step over its component function calls/loops one by one, inspecting the program state and diving into any that seem to be producing strange results. But overall, I definitely just find it's a lot more effective to start from a broad 'does everything look okay at this point?' than from a narrow 'I think the problem is this'.
- Swizec 10y ago
  > 'does everything look okay at this point?' than from a narrow 'I think the problem is this'. Even in a small real world codebase with just a few tens of thousands of lines and a couple hundred possible user interactions, you need the intuition for "I think the problem is this" or you're gonna be there for the rest of the year stepping through the code. Once you've scoped the problem to a potential few tens of functions, then you can spelunk through it to find the exact issue.

theaeolist 10y ago

Debugging an API, which may have obscure corner cases or usability quirks seems appropriate to me. "Debugging" an algorithm which has inherent logical flaws by attempting to patch it up with case-base reasoning and exceptions (logical, not runtime) is a road to disasters. I have seen too many devs trying to hill-climb their way out of a logical hole and the result is invariably a mess of a program which is still wrong. Also, in case of concurrent and parallel code debugging is basically useless since realistic timing conditions are not reproduced. I personally prefer logging, which gives a faster and more targeted trace of program behaviour than step debugging.

sleepychu 10y ago

My favorite debugging trick (when faced with an 'impossible' bug) is to state my assumptions, it's usually clear when they're all laid out which ones are conflicting.

mrSugar 10y ago
Sounds like a variation on Ayn Rand's "contradictions do not exist, if you think you see one, check your premises, some of them will be wrong".
- eriknstr 10y ago
  Is that quote intentionally saying that you have contradicting premises, and by extension is the statement in that sentence that contradictions do not exist contradicted by the later part of the sentence, or is there something about the sentence that I'm not understanding?
  - mysterydip 10y ago
    It reads like that at first. What it's saying is for example: Assume x is 5. Assume function TimesFive multiplies things by 5. When we do TimesFive(x) and my answer is 50 instead of the 25. I check the TimesFive function and it's just "print x*5". But how is that possible? 5x5 is 25! The problem in this case is my assumption that x would be 5 was wrong. I've had this issue before in my code (though not as trivial an example of course), but it's always a good reminder of something to check when troublshooting.
- dhodell 10y ago
  Which sounds like a variation on various Socratic learning techniques.
jspash 10y ago
Sounds like Rubber Duck Debugging. A simple, yet effective technique. http://wiki.c2.com/?RubberDucking http://wiki.c2.com/?RubberDucking https://blog.codinghorror.com/rubber-duck-problem-solving/ https://blog.codinghorror.com/rubber-duck-problem-solving/ Having just read the recent article on HN about Japanese workers "pointing at things" got me thinking that the two actions probably stimulate a similar area of the brain. Sometimes we just need to get "out of our head" to make a breakthrough in the problem we're trying to solve.

_nalply 10y ago

My secret sauce to debugging is formulating hypotheses about what went wrong then instrument the software to validate the hypotheses. By instrumenting I am talking about manipulating running software to give me information. Sometimes the manipulating step is made unnecessarily hard, and the trick is to force the hand. I see myself an inquisitor of software and use programmatic torture to get what I want to know. That's the central attitude to debugging. The devil is in the details, there are many possibilities to get information: using a runtime debugger, insert output statements, read the logs, use LD_PRELOAD, use strace, and so on.

mattmanser 10y ago

I personally found this to be a poor article on the subject. There's very little actual content about debugging, he spent more time talking about theories of intelligence than debugging. I've recently been thinking about this, thinking about making a course. It's my personal experience that his approach is flawed. Fundamentally, step one is not develop a theory. That is the worst thing you can do, it can waste a lot of time. I still sometimes guess what the problem is, but all I really use that for today is where to look and that's sometimes a waste of my time as I skipped step 1. Step 1 is recreate it. No theorizing, no thinking, no faffing around. Can you recreate the bug exactly? If you can't, find out why. Talk to the bug reporter, find out what they were doing, this is a people skill you need to learn as a programmer. Maybe you need access to the live data to see the state an object is in. There's only a tiny percentage of bugs (usually race conditions) that can't easily be recreated. If you can't recreate it, that's when you turn to logging to try and catch the conditions to recreate it, but logging is the last thing you should try and I rarely ever have to add it (we're talking 1% of bugs). I often see junior programmers littering code with log statements when trying to debug, I feel this is a bad method, but can understand the temptation, especially when they don't really understand what the code is doing (an understandable position I was in in my earlier days). Step 2 is isolate it. If the code is simple and it's obviously one method, no further work needed. If the code is complex, you need to really isolate the exact problem. Say you've got the wrong value displaying for a basket total, where is that coming from? Is it not updating, or is the total function the problem? Isolate the exact step in the process that's going wrong. Sometimes it's hard to isolate and logging is needed again, but same advice as before, logging is a rare necessity. Step 3 is make it easy to run a recreation. If you can run it in 10 secs or less, that's great. If it's one method of a complex web call, make a test endpoint that only calls that method that's failing and exactly recreates the conditions. Then you can run it quickly without all the cruft. In an extreme case, this can be taking the code completely out of the context and putting it in a tiny standalone app. Step 4 is read + understand the code. Is there an obvious logic bug? Don't jump to conclusions. Still don't get what's going on? Step through the code, breakpoints. Understand what's going wrong. Breakpoints and stepping is fine, inspecting variables, etc. Step 5 is fix it. This should be fairly simple now. This is also the time to think about the future. Is this part of the code a constant headache? Is it buggy because it's over-complicated? It might need a bit of a refactor to make sure this doesn't happen again. This is a judgement call, unnecessary refactors are expensive, might even introduce new bugs, but over-complicated code will lead to more bugs. It not being written in your favourite style is generally not a legitimate reason to refactor. Step 6 is testing the fix, recreating the conditions from step 1. Don't rely on QA if you have them. Ensure you've done your job. (Step 7 get rid of your logging statements + test it again! You just changed the code) Now you can skip steps, be far less formal about it. Sometimes it's obvious. Sometimes it's hard and you may need to iterate over 4/5/6 a few times. But never, ever skip steps 1 + 6. Recreate it. Test your fix. Because if you skip those steps, and you "fix" it, you don't actually know it's fixed.

jerianasmith 10y ago

Investigating is essentially a space particular term for critical thinking. Bugs are depicted as word problems.Since mental models are approximations, they are in some cases inaccurate, prompting unusual conduct in programming when it is created on top of flawed suspicions.

rdtsc 10y ago

To make debugging easier you have to pick tools (languages, IDEs, methodologies) that make it easier. In other words you optimize for debugging. For example, I find Erlang code very easy to debug. There are few reasons for that: * Built-in runtime tracing is easy. Can remsh into a VM node and trace any function at runtime. * Functional style with immutable data and variables make it easy to figure what is changing in the code. Because of immutability the state is always updated explicitly. That makes it much easier to understand what is happening for me that figuring it out than having a class inheritance hierarchy with methods overrides and such in an OO language * Code hot-patching makes it easier to test and iterate to zoom on in a bug. With care it can be done in production. You see strange failure you never saw before and which you had an extra log statement in there? That's easily done. * Language is simple and self-consistent. That means there is less chance someone used some obscure new feature I haven't heard about. This is a bit like C vs C++. C is very simple at the language level. You can still have hairy code with triple pointers, funky dispatch tables with Duff's Device thrown in there but at least you can read through it. Looking at C++ code I sometimes couldn't even parse what was happening because of a language feature (so they overrode the operator some place, and using a templates, ...). Or someone created a macro and then everything is more interesting all of a sudden. * Process heap isolation. When something happens with one process, I know for a fact that it hasn't scribbled on memory of other processes. I can restart that process if needed and test my hypothesis of what I think is wrong, often without affecting the rest of the system (even while it is running in production). * Sane concurrency primitives: Having the ability to create hundreds of thousands of processes with isolated memory often mean a smaller impedance mismatch with your business code so you have less total code to write. The less code your write, the less code you have to debug. Anyway the point being, debugging should be a trade-off in picking your tools. People don't think about it much and focus on other things first. But it will be something that over time will make a tremendous difference. > Virtual machine-based languages, interpreted languages, and languages with runtime environments encourage users to view the execution environment as a black box. The goal here is to make programming easier by reducing the scope of the mental model the programmer must maintain. When bugs occur in these execution environments, you're left with a complete gap in understanding. It can be the opposite as well. Having a VM with good tracing and debugging facilities is very useful. Often you can do things wouldn't be able to with just plain compiled code.

noyeda 10y ago

Nice article. Thought I'd note something: > Software developers spend 35-50 percent of their time validating and debugging software.1 This is a strong claim, so I looked at the cited source, given the unfortunate experience of having seen many strong claims backed by weak underlying data. > The research phase will predominantly comprise of short interviews with approximately 10 to 12 organisations that compile code on the LINUX operating environment. These organisations will also fill in the Cambridge Venture Project (CVP) survey to help quantify how much time they spend debugging with and without RDBs. Broader research in the form of the CVP survey with potential users of reversible debuggers will be conducted to gain insights on how much time is currently spent debugging. Further down: >49.9% Programming time spent debugging > According to 54 questionnaire responses to the CVP survey and 11 interviews, based on the question ‘Of the time spent programming, what percentage of your time is spent on the following: (1)fixing bugs (2)making code work (3)designing code (4) writing code.’ (1) and (2) then were grouped as debugging. While this might be indicative of a widespread phenomena, it doesn't seem to be enough evidence to support the factual and strong claim of the first sentence in the ACM article. There certainly isn't enough information in the cited source alone to decide (and given that, is this really sufficient as a source?). Perhaps I missed further evidence? That being said, I won't throw the baby out with the bathwater. Debugging is clearly a time sink in industry, regardless of what the actual percentage is. I enjoyed this piece overall. I was previously unfamiliar with some of these terms -- specifically incremental vs. entity mindsets -- and it shed some light on work issues I've seen (coincidentally in the debugging technology space). Being someone with an incremental mindset and having worked for someone with an unjustified entity mindset, I found the environment suffered from almost all the problems (and more) you described associated with the latter mindset. Granted, this is anecdotal. I quite strongly agree that improving the process and mindset behind problem-solving results in a significant improvement to specific skills like debugging software. It's refreshing to see an article like this instead of the many others hyperfocused on specific, less-abstract techniques (and widely-applicable abstract techniques/processes are the most effective use of one's learning time).

dhodell 10y ago
Thank you for the feedback. I chose (for better or worse) to cite the most recent article as opposed to maybe the least controversial one. Other papers I've read have suggested similar ranges; nobody presents a stddev so it's hard to compare. And how you classify "debugging-related tasks" is kind of subjective, as you correctly point out. So linking other papers might or might not be useful anyway. In "A framework and methodology for studying the causes of software errors in programming systems" (Ko & Meyers 2005), they cite a NIST publication as saying 70-80% of time is spent debugging. I parroted this off in a talk once, and then I looked at the same publication ("The Economic Impact of Inadequate Infrastructure for Software Testing" RTI 2002) and the only 80% I found was time spent in testing software in the early days of software engineering. In the 1990s, it finds coding and unit testing to comprise about 30% of the time. This is as far as I can tell, at least similar to the (1) and (2) in the CVP survey. But again, indirect comparisons, who knows? In tables 6-4 and 7-6, the RTI study finds different numbers for time spent on bugs, and different frequencies of where bugs end up being discovered. Whatever the rate, it is clear that post-production discovery bugs are the hardest to fix, and I think this is because you only ship code when you have high confidence in its correctness. I think this is a hard measurement to make. It's hard to find participants, hard to guarantee they have the same idea of what "debugging means", hard to quantify. So thank you for calling me out on this. But I think other studies have evidence that this isn't too far off from a correct measurement. For example, Gugerty and Olson in 1986 did some novice vs expert study at debugging LOGO programs. (I don't like the Dreyfus model for classifying debugging, but whatever.) They found that novices took about 18 minutes to fix bugs, testing an average of 3.6 hypotheses per program, with the first hypothesis being correct only 21% of the time, and re-executing code every ~3 minutes. Experts took 7 minutes, testing an average of 1.6 hypotheses per program, with the first hypothesis being correct 56% of the time, re-executing code every 2 minutes. So guesses are only right maybe half the time if you're really good. Basically any novice vs expert study compares how much time people spend on the problem (they're trying to reduce debugging time), but it's hard to extrapolate this into percentage of development time. For example, some people spend 100% of their time debugging because they're on QA or sustaining engineering teams. To be quite honest, there _fundamentally is not_ enough research into the practice and pedagogy of debugging. I have about 60 papers in my "Debugging Papers" folder that I've read over the past year, and basically everything up until the Dweck-inspired research in 2008 isn't great. There are only 9 papers in all of computer science that cite Dweck that I've found as of maybe 6 months ago (I'm not a researcher and I don't look that often). From memory, two of them were recommending further research based on her work, maybe five of them were from people in CS departments helping Dweck create games and programs to perform her research, and then only two were actually applying her work to debugging. I would agree with you if you said this needs more research, and that the claim isn't strongly supported by the provided evidence. I greatly appreciate your feedback.

sn9 10y ago

Udacity actually has a course on systematic debugging taught by the creator of DDD. The course covers the importance of the scientific method and hypothesis testing, divide and conquer approaches, etc.

edejong 10y ago

Of course, Dijkstra had his view on it. I try to adhere to his position and prefer proof by type-checking over proof by single example. "A common approach to get a program correct is called "debugging" and when the most patent bugs have been found and removed one tries to raise the confidence level further by subjecting the program to numerous test cases. From the failures around us we can derive ample evidence that this approach is inadequate. To remedy the situation it has been suggested that what we really need are "automatic test case generators" by which the pieces of program to be validated can be exercised still more extensively. But will this really help? I don't think so." [1] Linus (used) to have a similar position [2] Debugging hides the underlying architectural problem: to either communicate or abstract the algorithm in such a way that it can be understood by the programmer. If something needs debugging, not the bug needs to be fixed, but the architecture to understand the bug before debugging. [1] Edsger Wybe Dijkstra - EWD303. https://www.cs.utexas.edu/users/EWD/transcriptions/EWD03xx/EWD303.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD03xx/E... [2] http://lwn.net/2000/0914/a/lt-debugger.php3 http://lwn.net/2000/0914/a/lt-debugger.php3

dhodell 10y ago
Thanks for this. This seems to be basically what I'm saying through most of the article. Bugs occur because of a gap in a mental model. To fix the gap, we need more understanding. I'm not sure I would say that's necessarily a design or architecture flaw in the software. Unfortunately, proving things correct in most popular languages is a Very Hard Problem. I do address that to some extent in the article, though. In particular, the sorts of languages that end up being easy to prove correct tend to be difficult for people to learn. Therefore, people are likely to either learn something easier, or they are likely to write code that is "correct" or "safe" but still has logic errors. I think that there are other many valid reasons to write software, and not all of them require correctness to the extent Dijkstra would like. But I think that's a pipe dream anyway. For one, most software is not written in formally verifiable languages, or formally verifiable dialects of popular languages. Porting all software would be impossible. Second, who defines "correct"? What about bug-compatible software, where the definition of correct is imitating something else that is known to be incorrect? What about ambiguity in specifications like JSON, or undefined behavior in C (some of which can only be detected at runtime, because we don't actually have infinite tape in our Turing machines)? What happens when there's a bug in the runtime of your formally verifiable language? What happens when there's a hardware bug? I'm not saying these are necessarily common, but regardless of your software environment, you're running on something else that can have the same bugs. People make mistakes, even when proving things. Math is great, but people make mistakes, and our tools are still bounded by the laws of physics. No matter which way I look at this thing, I keep coming back to teaching and understanding as the only generalizable and approachable ways to address the problems bugs. (Whether that approach is taken by proving correctness of some implementation, or post-hoc with testing and tools.)