21 ms·

Debuggers Suck

• sfink 7y ago
  I will corroborate one main point of the article: debugging with rr is so much better than a traditional debugger that I will only buy Intel hardware, even with all of the security flaws and corresponding performance hits, even though recent AMD CPUs would halve my half-hour compile time. It really is that much better. Once you start using it, it's really hard to go back to a situation where you don't have it. It's like driving a car that can only turn left -- sure, if you need to go right you can always 270 degrees to the left (by rerunning over and over again to get closer to what you want to examine, assuming your problem is reproducible enough), but it feels burdensome and ridiculous. If AMD fixed their performance counters so that rr could work, I would switch immediately. I am also a fan of Pernosco. It is especially useful when hooked up to a continuous integration system that can hand you a ready-to-go recording of all failures. You can get assigned a bug with a recording to go along with it. Even if it doesn't look important or tractable to look into the ordinary way, having a recording makes it easy enough to take a glance and check it out that you'll actually do it. At shops that do a lot of system-level programming (as opposed to scripting), the productivity boost ought to be worth a serious amount to the bean counters.
  • blub 7y ago
    I guess if it doesn't even work on AMD, ARM must be out of the question...
    • fabrice_d 7y ago
      Technically possible, if someone wants to sponsor: https://twitter.com/rocallahan/status/1199477451602067456 https://twitter.com/rocallahan/status/1199477451602067456
    • bzbarsky 7y ago
      https://github.com/mozilla/rr/issues/1373 https://github.com/mozilla/rr/issues/1373 is tracking that, but the short story is that ARM seems to not provide enough performance counters to make the current rr approach work at the moment. So it would take either CPU-side changes or a somewhat different approach.
  • archgoon 7y ago
    What prevents you from having a build server that uses AMD cpus? Budget or physical space limitations? Power consumption concerns? Too complicated to have different build and test boxes? Large binary and debug symbols that would result in no effective gains after network transfer time was accounted for?
    • sfink 7y ago
      In theory, nothing, and that's a good idea that I've considered. My past experience with distributed builds (distcc, icecc) hasn't been that great. They've worked, but they tended to break down or slow down, and I ended up spending more time maintaining my setup than I gained in compile times. Perhaps things have improved. I still have bad memories of trying to diagnose why the network transfers had slowed to a crawl again (on a local gigabit network.) The other demotivator is noise. I work from home, and the other family members who share my office aren't keen on the sound of a fully spun up build server. I could run ethernet cable under the house to another room, maybe. (Relying on wifi bandwidth hasn't worked out very well. If only debuginfo weren't so enormous...)
      • vardump 7y ago
        > network transfers had slowed to a crawl again (on a local gigabit network.) 10 gigabit is pretty cheap nowadays. Just in case your problem could simply be solved with higher bandwidth...
        • sfink 7y ago
          It very rarely hit the actual bandwidth limit. It would start out close to it for a while, then drop down. And down. And down. Until it was using like 2% of the full bandwidth, but never completely stalling.
  • glandium 7y ago
    FWIW, rr support for AMD CPUs is getting close. See https://github.com/mozilla/rr/issues/2034#issuecomment-556494535 https://github.com/mozilla/rr/issues/2034#issuecomment-55649...
    • garaetjjte 7y ago
      I don't see any signs here that it is close, just that it is still broken in the same way.
      • fulafel 7y ago
        Also from the issue: "the bug, whatever it is, could be in the kernel and not the hardware"
  • jchw 7y ago
    I will concede that rr is likely extremely useful, but as a counterpoint to this and the entire article, getting bugs earlier and earlier in the development cycle beats better debugging any day. I will definitely try to add rr to my toolbelt if Ryzen support ever lands, but I still prefer catching problems by writing fast test cases instead. It's like debugging that happens automatically.
    • olodus 7y ago
      I don't see these as separate solutions. Running your tests in a debugger usually makes it so much faster and easier to find where it went wrong. I would argue find bugs at compilation would be a separate solution from this and even earlier in the dev cycle.
      • jchw 7y ago
        Indeed, finding bugs at compilation is better, which is why I prefer languages with strong typing. I reread my comment to make sure, but I am being painfully explicit that good debug tools are still useful. But I prefer good tests over even good debug tools. I very rarely even printf debug as a result of automated testing. This isn’t really just lip service; I have a project right now where I literally can’t attach a debugger or run locally. Of course, that’s a flaw and unreasonable. But, with good test hygiene, you can still be pretty productive in the face of this. Not only that, pushing code feels safe, even without the ability to actually run the code locally. Do I think debugging is bad? No. I think it’s worse than testing, which when done right can save you from a hell of a lot of manual debugging, even in the face of complex software with networked dependencies. I get ire on HN every single time I mention the virtues of testing instead of debugging. I suspect people literally don’t believe you can exchange quantities of one for the other, but where I’m standing it’s obviously possible. Same thing comes up with strong typing frequently, where you have people making the claim that typings don’t prevent bugs - but anyone who’s transitioned a medium sized code base to TypeScript can attest to the improvements it can bring. In my opinion, automated tests are almost literally just debugging done by a script instead of a human. edit: It's also being pointed out to me that rr is being used to debug failing tests, which is not something I thought of and my ignorance may be contributing to some of the reason my comment was so poorly received here rather than the testing vs debugging comparison. (In many cases, for unit tests, I had never thought of using a debugger to try to find the problem, because the actual problem is often made obvious enough by the failed assertion output.)
  • de_watcher 7y ago
    The wild thing is that scripting languages don't have that kind of a debugger.
    • glandium 7y ago
      Javascript does, with caveats. https://developer.mozilla.org/en-US/docs/Mozilla/Projects/WebReplay https://developer.mozilla.org/en-US/docs/Mozilla/Projects/We...
      • The_rationalist 7y ago
        Is there any equivalent on chromium? I believe Firefox devtools have been ported to chromium.
        • sanxiyn 7y ago
          As far as I know, there isn't. This needs substantial support in the engine, so "devtools have been ported to chromium" is not enough.
    • yvdriess 7y ago
      Depends. Image-based runtime languages such as Smalltalk and various Lisps will, crudely put, pause at the site of the error and give you options to fix it and continue. That already covers 99% of the OP's issues with gdb. edit: Implied is that image-based runtimes by definition produce reproducible snapshots of the error, with the full power of the Lisp or Smalltalk introspection tools. Edit-continue is the icing.
      • sanxiyn 7y ago
        Not at all. Edit and continue is only a small part of what debuggers can be.
      • vidarh 7y ago
        Doesn't need an image based runtime either. Ruby's "pry" is inspired by Smalltalk, in that it's providing functionality to manipulate the running environment, edit and reload code etc., and continue. During development I almost always have a make target to drop me in a pry session with all the app code loaded and initialized so I can play with the code as it is actually used.
  • srcmap 7y ago
    I have use gdb and various debuggers for a long time. They are very useful but there are limitations. "gdb rr" - tried it with demo helloworld. That works. But when tried it with a more complex program (FBOSS - Faceboss' Switch/Router open source program), gdb rr function core dump immediately. Also there are certain categories of bugs what are very difficult to use gdb with: * multi threads / race condition bugs - gdb break point in affect the flow of the program and cause different behaviors. * Bugs can not be reliably reproduced. * Server programs or embedded system programs where the code can not be stopped. * This problem get much worst when combine with performance related issue - I work on one system design requirement that would switch over to backup system when system missing a heart beat for 50 milliseconds. Any gdb breakpoint in that app would automatically trigger a hard system fail over to backup. BTW, I do use gdb a lot and can script gdb to do conditional breakpoints with script that auto dump variables, info etc.
    • rhinoceraptor 7y ago
      I would highly recommend looking into dtrace or bpftrace (depending on your platform). They're not as powerful as a debugger, but you can use them when nothing else will work. They're totally safe to use in production, because they can't modify memory, and they have a very small performance impact when in use.
    • bzbarsky 7y ago
      I'm not sure what you mean by "gdb rr". "rr" is a separate debugger that uses the gdb frontend but a different backend. Importantly, it does not utilize breakpoints during the initial program execution, precisely to address your first, second, and fourth points. The third point is still a problem, since it requires the program being debugged to run under rr from the start.
    • roca 7y ago
      If you really want to use rr on FBOSS, please file an rr issue and we'll look into it.
• KenoFischer 7y ago
  Another testimonial from me. I basically refuse to do any debugging on systems that rr doesn't run on unless there's a very good reason. I use rr extensively whenever there's a bug in Julia and I would singlehandedly credit it with a measurable increase in system stability due to it catching bugs we'd never have gotten otherwise. roc is one of the few people with both the vision and persistence to really improve the debugging experience.
  • mrspatula 7y ago
    What's the best way to use rr to debug Julia code atm? Or are you talking about debugging the actual language code in C?
    • KenoFischer 7y ago
      Yes, the latter (mostly for various weird corner cases). Back in the Gallium.jl days we could actually debug the recorded julia code, but we haven't regained that capability yet.
• BubRoss 7y ago
  I agree but this whole thing just says 'debugging sucks, my commercial tool is great, I won't explain a single detail about it, follow this link' (it is a wrapper around rr). He's preaching to the choir and I still think this post is sleazy. Also replay debugging doesn't have to be complicated. This enables replaying sections of a program with hot reloading by requiring that the data structures are serialized. It also catches low level exceptions so that replay and hot reloading won't crash the running program. https://github.com/LiveAsynchronousVisualizedArchitecture/lava https://github.com/LiveAsynchronousVisualizedArchitecture/la...
  • heycam 7y ago
    There's a great series of posts going into more detail about Pernosco's design and advantages that is linked to in the final paragraph: https://pernos.co/about/ https://pernos.co/about/
• mc3 7y ago
  Unit tests make debugging suck a bit lest, because of the ability to change the test case (on local machine only!) / and tested code iteratively in combination with the debugger to find the source of the problem, with fast feedback cycle. Debugging using the UI is another tool. Advantage is you can quickly cover new scenarios and change what you do based on new info. Downside is loading time can be a lot longer than running a unit test.
• bsder 7y ago
  As much as I love rants like these, the problem with debuggers is discoverability. I have lost track of the number of times that I kill myself debugging something with gdb only to later explain it to some group and somebody says: "LOL, n00b. Why didn't you just use <magic gdb command>?" Why? Because I didn't know about the magic gdb command and there was no documentation anywhere that would have pointed me to the fact as to that command being exactly what I was busting my head over. Debuggers are like build systems. You don't diddle with them every day and when you are something is wrong and you're on a deadline. If people want programmers to appreciate better debugging, we need some good videos of people debugging real problems in real time with debuggers.
  • db48x 7y ago
    https://www.youtube.com/watch?v=LR0pms_mQuY https://www.youtube.com/watch?v=LR0pms_mQuY was nice to see
  • TeMPOraL 7y ago
    Speaking of GDB and arcane commands - GDB does have decent documentation. It actually ships with a whole book on how to use it. Try `info gdb` in your terminal (if you get a man page, you're missing info pages; on Ubuntu/Debian, you have to install gdb-doc package). One other way developers suck (myself too often included) is that they don't RTFM. And I don't mean "skim the manpage" RTFM, but actually read the included manual. If you're going to use something day in, day out, it's really worth it to spend the couple of hours required and read the included manual cover-to-cover. It saves a lot of time and frustration in the long run. (I don't use GDB myself all that much, but the last time I had to, I got tired of flailing around; I read the info page cover to cover, and the time investment repaid itself on that single task.)
  • roca 7y ago
    OP here. I get it. I led rr development and even I sometimes find gdb features I never knew about! We explicitly tried hard to make Pernosco discoverable, partly by creating simple but powerful and composable features, and partly using traditional GUI techniques. We definitely haven't fully succeeded, but I think we moved the needle. If you dig through https://pernos.co/about/ https://pernos.co/about/ you can see what we tried, and if you try the demo https://pernos.co/debug/e9UFXkGq__8KQB-N6n59lA/index.html https://pernos.co/debug/e9UFXkGq__8KQB-N6n59lA/index.html you can see how well we succeeded.
• mantap 7y ago
  It would be amazing if someone could design a programming language around debugging. There are many languages that are designed around preemptively catching bugs using a type system, and I do use static type systems, but this in no way oblivates the need for a debugger. The way I want to write code is directly as it is executing. I think the distinction between writing code in an editor and debugging is arbitrary - I want to edit programs as they are running and see the data as it flows through beside the code. I want to be able to browse the object graph as if it were a filesystem like I could in Objective-C (I think Smalltalk invented this?). This does require a different code architecture to be effective: more pure functions and less tangled state, but those are usually considered to be good things. Improving debugging might have a positive effect on code quality.
  • Stratoscope 7y ago
    > The way I want to write code is directly as it is executing. I think the distinction between writing code in an editor and debugging is arbitrary - I want to edit programs as they are running and see the data as it flows through beside the code. I do something like that every day, especially when I'm dealing with weird and arbitrary APIs (which is most of them). I write as much of a function as I know how to do, set a breakpoint on the next line, and run what I have until it hits that breakpoint. Then I can write the next few lines of code while looking at actual data. If it happens to be Python code and I'm using IntelliJ, I can also open a Python console - a REPL running in the context of my breakpoint. This helps me understand the libraries and APIs I'm working with in a way that just editing static non-running code doesn't.
  • m463 7y ago
    > design a programming language around debugging I would like if existing languages allowed you to "level up" your code. examples that have worked for me: "use strict" in perl, "tainting" in perl, assertions in C (runtime and compile time), -Wall -Werror in shared codebases, and similar. the idea would be to incrementally make everybody's code a little better in manageable increments.
  • extrapickles 7y ago
    Visual Studio lets you do some of this in a clunky way with ‘Edit and continue’, where you can edit code after a breakpoint in the same method. I believe most browser JS debuggers let you edit the JavaScript, though it can be quite buggy.
• sesuximo 7y ago
  FWIW gdb can record (sometimes) https://sourceware.org/gdb/wiki/ProcessRecord/Tutorial https://sourceware.org/gdb/wiki/ProcessRecord/Tutorial
  • sfink 7y ago
    Plan to spend a lot of time first narrowing down the time window of interest so it doesn't overflow its recording buffer, and then waiting for it to run at 1/100 speed. I used to use it. It was so painful and limited that I think it helped only a single time, and was all in all a net loss in productivity.
• jnwatson 7y ago
  I used to sell debuggers a decade ago. We even had reversible debugging. The amount of personal development productivity I had was unmatched; I’ve never gotten back to that point. It is true, nobody wants to pay for engineering tools. I wish the author luck.
  • m463 7y ago
    The market is so small, and there appears to be lots of premature optimization of the checkbook. I remember many years ago trying out purify, and it revolutionized my debugging. The problem seemed to be that the program I wrote could be debugged by everyone - if you just got everyone a per-user license. It would have been hard enough to justify a license for me (a person at the bottom with no power), it was impossible to justify a license for everybody.
    • TeMPOraL 7y ago
      > It would have been hard enough to justify a license for me (a person at the bottom with no power), it was impossible to justify a license for everybody. "Floating" licenses are better here. In my experience, companies are reluctant to buy licenses for an entire team for something that's used only occasionally. It's easier to make them buy a few floating licenses to be shared by the team.
      • m463 7y ago
        I am uncertain if floating licenses were available at the time. Even so, when floating licenses became popular, there was also the "license in use" denial problem. Unfortunately there's another way to do this nowadays. Offer convenient tools with analytics, etc... (and coincidentally spy on everyone)
  • Mathnerd314 7y ago
    Google has some internal IDE, presumably maintained by a small (paid) team: https://news.ycombinator.com/item?id=12842284 https://news.ycombinator.com/item?id=12842284 IDA Pro has been doing alright: https://www.hex-rays.com/products/ida/order.shtml https://www.hex-rays.com/products/ida/order.shtml But it's definitely hard to compete with open-source alternatives, piracy, general skepticism, etc.
• jpochtar 7y ago
  As someone who's also worked on a debugger, I'm sad to say many programmers aren't working on problems that needed complex debugging. I couldn't understand why friends weren't interested in what I was working on, until I realized they were building CRUD apps all day. There's little control flow to speak of in there, so if they had a problem, they could just dump locals() to get a good view of everything that happened. When I went back to doing compilers work, I missed my debugger dearly. I also realized why I needed a debugger and my friends didn't: debuggers like this are particularly useful for work with compiler-like characteristics, of which there is unfortunately not enough.
  • rtpg 7y ago
    I spend my day on CRUD apps and disagree. Often big CRUD apps with a lot of business logic tend to have higher level, "logic flaws" that generate bugs. In those cases having something like `rr` to trace the origin of certain values would be a godsend. I think these apps tend to be very wide in breadth, where instrumenting the entire system can be hard. Languages like Python (with `pdb` being a single-line in the middle of a program) are great in these scenarios because it allows for a pretty exploratory strategy when debugging. Unfortunately there's a lot of people who do "print statement debugging" because the tooling really isn't practical in most places, but it would definitely be useful.
    • hinkley 7y ago
      In general, I find a sense of learned helplessness among coworkers. They don't feel like they can get much useful out of the debugger. In a way they are often right, but in part because it's a self-fulfilling prophecy. If you don't use the debugger much, then excessive amount of indirection in many codebases is academic to you, the way (unfortunately) people think of color blindness, accessibility, hell internationalization for some. It's abstract, not something you have to deal with all the time. It doesn't affect you personally so you just ignore it (in other words, complete lack of empathy). Once the codebase is full of shared mutable state, lasagna code, lavaflow antipatterns, callbacks, handlers, etc, then it really does seem like it's easier to pepper some print statements around and pray. It's coherent, but wrong. Like a teenager's justification for how they deal with their messy room. Your life really would be easier and healthier if you cleaned some of this shit up, man.
• aleister_777 7y ago
  You're probably right, but no one will believe you. Sorry.
• loupeabody 7y ago
  just gonna leave this here RemedyBG for Win64: https://remedybg.itch.io/remedybg https://remedybg.itch.io/remedybg
• xanth 7y ago
  Has anything like RR been developed for the .net Core CLR or Framework?
  • james_s_tayler 7y ago
    I was under the impression the Enterprise version of visual studio includes a debugger that functions in a similar manner. Capture the entire execution and allow it to be replayed and shared with others.
    • imstuff 7y ago
      It seems it's only for Azure: https://docs.microsoft.com/en-us/visualstudio/debugger/debug-live-azure-virtual-machines-time-travel-debugging?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/debugger/debug...
  • kevingadd 7y ago
    This claims to provide replay debugging for C# and C++, but I haven't used it: https://docs.microsoft.com/en-us/visualstudio/debugger/intellitrace?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/debugger/intel...
  • ynik 7y ago
    WinDbg Preview's Time Travel Debugger can do this: https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/time-travel-debugging-overview https://docs.microsoft.com/en-us/windows-hardware/drivers/de... The great advantage of such a debugger is that you can use it to follow data flow: if you see an incorrect value somewhere, you can set a memory breakpoint and then run the program backwards to jump to the point in time where the value was written. I've only used it with C++ applications so far, but the WinDbg blog claims that it also works when debugging managed code with the SOS debugging extension. Visual Studio confusingly has the somewhat-similar "Intellitrace", but AFAIK that only records previous states collected for certain events (breakpoints, first-chance-exceptions, ...). It can't rewind the program an arbitrary point like the time-travel debugger can, so it's not useful to follow data flow.
• forgotmypw3 7y ago
  Possibly controversial opinion: The best debugger I ever used was VB3-6, and same-era VBA.
• xwdv 7y ago
  Tragically, one of my regrets in life is never really learning proper debugging. It is embarrassing that even as a senior engineer in the industry, the vast majority of my debugging is basically a bunch of console logs, and it gives me a serious case of imposter syndrome when I see others using all sorts of fancy debugging tools. If there is a silver lining, it must be that my lack of debugger skills can only mean I spend a lot of time writing safe, bug free code. Or more likely, no code at all.
  • sfink 7y ago
    Fixing bugs without a debugger is a great skill. If you're good at it, you'll be able to figure out many problems far faster than people who depend on debuggers, and you'll be more likely to get to root causes and change things to be more robust at an architectural level. Fixing bugs with a debugger is a great skill. If you're good at it, you'll be able to figure out many problems far faster than people who don't use debuggers. You can skip a lot of deep thinking and go straight to the site of a problem. You can explore the actual execution paths in ways you'd never figure out from all the layers of templates and overloads. You can debug other people's code almost as easily as your own. You won't get caught by some subtly wrong assumption you're making when thinking through how things are supposed to work. Debuggers are dangerous. They're insanely useful, but once you start depending on them, your ability to fix things without a debugger will atrophy. If I had the self-discipline, I'd ban myself from using debuggers for a month out of every year.
• devin 7y ago
  I do almost all of my work in Clojure and have for nearly 7 years. I don’t actually miss a traditional debugger much. I’ve occasionally tried one out, but an interactive REPL and immutable data means printlns with the occasional conditional to sample an expression are surprisingly pretty good. If you add generative testing via spec or test.generative, you can simulate lots of the scenarios you’d otherwise be desperate to have a dump of the environment to inspect. If you get there, profilers usually tell the story in adequate detail. Profiling and debugging are related and often share responsibilities.
• theamk 7y ago
  > In particular, developer culture is that developers don't pay for tools, especially not debuggers. I blame larger programming teams. It’s a chicken and egg problem: if you want to start using a commercial debugger, you need to buy it for everyone on dev team, so you can share your knowledge and benefit from others’ help; and you are not going to buy a debugger for everyone if you are not using it yourself.
• pjmlp 7y ago
  Maybe I missed something, but what really sucks is making "how to use a debugger" demos, because most developers don't even know what their debugger can do for them, let alone use it properly. Visual Studio team started doing VS productivity talks, because they noticed that many feature requests were about features that VS already supports for ages.
  • googleisevil6 7y ago
    Microsoft sucks. When it comes to telling people about features. I hope they are using telemetry to fire people. If your feature is unused and unwanted- engineer is fired. If your feature is unused but wanted- a marketer is fired.
  • int_19h 7y ago
    It can be worse. I work on Python debugger in Visual Studio and VS Code, and it's still unbelievable just how often, when I describe what I do to people who write production code in Python, I have to explain what e.g. conditional breakpoints even are, and then they say things like, "Wait, there is a tool that can do that? That could have saved me a few hours the other day!". And this all has been around for many years now... There's some progress - these days it's mostly confined to the data science crowd, and web devs have mostly caught up, due to the rapidly increasing popularity of PyCharm and VS Code over the past few years. In general, it feels like there's some weird gulf when it comes to tooling. On one side, you have the hardcore C gurus (often doing embedded), who do heavy scripting in gdb to solve very complex problems. On the other side, you have IDEs that grew out of 1990s RAD - Eclipse, Visual Studio etc - in ecosystems where the language and the tooling are usually developed and packaged together (e.g. whenever a C# compiler ships a new language feature in stable, there's also a new stable version of VS that supports it in the editor, the debugger etc - and UX for those is explicitly considered when designing features). And in the middle, you have all the high-level "scripting" languages, which have both low-level debuggers and powerful high-level IDEs - but their respective dev cultures largely ignore both for historical reasons.
    • mbeex 7y ago
      There is more to it. In the past, I was a all-day VS User (C++ most of the time, but even Python. Today I prefer Wing IDE for this (and sometimes VS Code)). While conditional breakpoints can be very useful, they often are not (especially because they slow down the code - hitting a variable value after 20000 iterations in a heavy-loaded loop as an example). It follows, for most developers syntax doesnt sink in. At least in the past, documentation was too far away. These things have to be literally at your finger tips. Don't know about the situation nowadays, though.
• modeless 7y ago
  I dearly wish that I could use rr and Pernosco, but I do graphics work and GPUs don't work with fancy debuggers. The state of GPU debugging is even more dire than CPU debugging.
  • ensiferum 7y ago
    Have you tried NVIDIA NSight Graphics ? If you're on NVIDIA hardware this really works quite nicely and supports OpenGL, DX and Vulkan.
  • uonyx 7y ago
    A debugger like this (sort of) already exists for GPU work (but for Metal). https://developer.apple.com/documentation/metal/shader_authoring/developing_and_debugging_metal_shaders https://developer.apple.com/documentation/metal/shader_autho... https://developer.apple.com/videos/play/wwdc2018/608/ https://developer.apple.com/videos/play/wwdc2018/608/
• jdblair 7y ago
  There's no mention here of inspecting core files, something that very much allows you to inspect the state of a program when the bug occurred, but after the fact. Is this because inspecting core files also sucks, since it's a snapshot of a point in time, and it's no use if your stack is corrupted? Or is it because this is a lost art? In my embedded work I avoided using debuggers because setting up remote debugging can be a PITA. I figured out a long time ago that building remote debug setup into my workflow will save oodles of time in the end. Still, I find myself using plain old logging more often than not - the bug may not be in C or C++, it may be on the JavaScript run in the interpreter embedded in that C++ application.
  • gmueckl 7y ago
    Core dumps are great, especially for the occasional odd bug that occurs only in long running processes. However, the ridiculous systemd default of attempting and failing to capture the kernel core dump into the binary journal completely ruins this. Only small, uninteresting core dumps are ever captured; large ones are silently discarded because the journal is not allowed to grow that big.
    • sfink 7y ago
      Yeah, when I'm crashing and want to keep cores, I'll temporarily modify /proc/sys/kernel/core_pattern to prevent systemd from eating them. I sometimes want them suppressed, sometimes want them written out, and never want them to go to systemd.
      • Pahr3yah 7y ago
        One issue is that that sysctl is not a namespaced one.
  • clarry 7y ago
    > Is this because inspecting core files also sucks, since it's a snapshot of a point in time Yes. Something is wrong, but you can't tell how you got into that state.
• slavik81 7y ago
  My personal pain debugging C++ with GDB recently has been the difficulty of creating objects. I spent a long time trying to figure out how to call operator[](vec2u) on my custom data structure. In the end, I gave up and built self-introspection tools directly into the program itself. It may not be a general solution, but it worked for me.
• thayne 7y ago
  > In particular, developer culture is that developers don't pay for tools, especially not debuggers I think there are a couple of reasons for this. One is that for many if not most developer tools, there are free options that are at least competitive with paid tools. Another is that developers can, at least in theory, read source code to better understand a tool they use, or even improve it, and therefore want access to the source code. Which means open source tools have an advantage over proprietary tools in addition to price.
  • DC-3 7y ago
    This line galled me too as it showed a serious misunderstanding of the principles of libre software. There's a reason we say Free as in Free Speech, not Free Beer! Why would I ever want to make my workflow dependant on a third party blob whose internals are obscure from me?
    • leoedin 7y ago
      The price does also play a role. Learning tools and frameworks takes a lot of time, and if it's not a free tool then if you change jobs or even departments you might find yourself unable to get a license. If you want to work on stuff at home, you need to pay. If you want to run a test server you need a license. Some tools are better than others in that regard, but it's basically why I would always choose learning a slightly worse free option than a paid option.
      • thayne 7y ago
        Oh i wasn't saying that cost doesn't matter, just that it isn't the only factor.
    • roca 7y ago
      One thing about debugging is that there isn't much lock-in. If you don't like the debugger or it goes away, switching to another debugging approach is very easy. Your workflow doesn't have a hard dependency on it (other than to the extent you enjoy using it). It's a bit like Github, which also isn't free software, but so many free software projects use it, in part because people think they could easily switch away from it if they wanted to. (I think in practice switching away from Github would far harder than switching debuggers.)
• hannofcart 7y ago
  > TL;DR Debuggers suck, not using a debugger sucks, and you suck. While at the outset this post seems like a rant, I believe the opinion expressed is very astute. In a perfect world, we would all write programs as a composition of mostly small pure functions, with type systems that can enforce provable correctness, and a small part of the code would generate 'effects'. We don't live in such a world. We often inherit gargantuan codebases written in an extremely imperative fashion, with a lot of state management baked into functions thousands of lines long with conditional and loop blocks nesting 5 levels deep. And there's a bug in one of those loops. A good step debugger that we could reverse step and inspect would save us somewhat in such times of perils. But they don't really exist. Or if they do, don't integrate with your editor. So needing debuggers suck. Debuggers themselves suck. Developers suck. Deadlines suck. Sigh.
  • sanxiyn 7y ago
    The whole point of this post is that such a debugger exists, and that you can use it, right now.
• sansnomme 7y ago
  A lot of debugging UX is editor integration. For top end IDEs, these are available out of the box. Set breakpoint by clicking left margin of any kind of code, another click to run trigger debugging. Similar story for VS Code though a bit more configuration is required. It's only an issue when it comes to lightweights like Sublime Text/Emacs/Vim. The philosophy of things being "loosely coupled" means that you have to be prepared to spend an hour browsing man pages just to figure out how to create a breakpoint. Edit and continue isn't just a fancy gimmick used in Lisp; it's a first class feature in Visual Studio (iirc it originated in the original Visual Basic, got removed during the transition to .NET before being added back again to Visual Studio, someone from MSFT should confirm this) Python is just "rediscovering" how to do this. JavaScript (in Chrome at least) have partial support, mostly because they were forced to do livereloading so frontend development isn't too unpleasant. Rust and Go and all the new age languages never seemed to have heard of it despite proclaiming their commitment to good developer debugging process (Elm might have some support though their blog releases seemed more interested in time traveling functional data structures rather than benefits to developer experience, and no, you do not need fancy stateless functions to add Edit and Continue though I suppose they do make it easier).
  • jng 7y ago
    Edit and Continue appeared in VS 4, 5 or 6 (can’t remember which one exactly), right before the first VS for .NET which I think was released in 2002. It supported C/C++ out of the box and required quite complex linker plumbing. Second .NET VS was 7.1, releases in 2003, before MS switched to confusing year-versioning in user-facing contexts and number versioning internally (including installation directories, project files, etc), which is made way worse because these days internal versions and years are off by two or three units (ie, Vs2018 is internally version 16 or 17, can never remember). My commercial vi/vim emulator, ViEmu, supported VS.NET 7.1/2003 on its initial release in 2005, and I still maintain that build, as in I build for that target every new ViEmu release and occasionally use it (it’s a much leaner and more stable debugging environment than more recent VS versions).
    • Crinus 7y ago
      Note that with the edit and continue appearing in VB he meant the immediate window in VB1, not the C++ feature. Also this didn't really appear with VB1, it most likely "appeared" with the first BASIC Bill Gates wrote (as far as Microsoft is concerned, BASIC had it before MS) as any number-based BASIC was able to both run and edit code at the same time and this was evolved to QuickBASIC/QBASIC's "immediate" pane at the bottom where you could run functions as soon as defined them in the editor and modify the program's code and state by pausing it and then this was passed on to VB1 though a dedicated window that remained there until VB6.
• comex 7y ago
  I‘d be willing to pay for a better debugger. But if the debugger doesn’t come with source code that I can inspect and modify, that’s a big negative that will have to compete with whatever advantages the debugger has. (It’s not just a hypothetical use case. I’ve grepped through the source code of GDB and LLDB on several occasions to figure out why they weren’t working properly.) And if the debugger is SaaS, meaning it will almost certainly disappear in a few years, after I’ve gotten used to it… it would have to be some super magic unicorn stuff for me to consider touching it, even if it were free. Pernosco is both closed-source and SaaS. Pass.
  • sfink 7y ago
    Try out rr first. It has at least half the magical unicorn sparkles you're looking for, is open source, and runs locally. It's not hard to modify. (I've done it, though not for any core functionality changes.) Then you'll be in a far better position to evaluate the additional benefits Pernosco might provide you. I think your reasons for being leery of it are valid, but there are very legitimate scenarios where it makes sense IMO. (Recording CI jobs automatically, collaborative debugging, situations where the critical dataflow is hard to follow...) And they integrate, too - you can upload rr recordings to Pernosco when you need the extra functionality. I guess it also depends on the pricing model, which I know nothing about.
  • sanxiyn 7y ago
    If you aren't already using rr, which is open source, do try it. And yes, Pernosco is super magic unicorn stuff. Do read about it.
• deleted 7y ago
  [deleted]
• deleted 7y ago
  [deleted]
• ipoopatwork 7y ago
  It's probably the only part where hardware development (FPGA or ASIC) has a significant edge: "time travel debugging" is the norm, where you have a wave window with your design state cycle by cycle. Can't wait to see more of that in software!
• orisho 7y ago
  Disclaimer: I'm a senior dev at rookout.com, which does something similar to Pernosco/rr but tailored to languages used for backend/web like Node.js, Java and Python instead of statically compiled (to machine language) languages like C/C++. I definitely agree - I've watched people painstakingly stare at code, add print/log statements and waste enormous amounts of time -- and then I asked them why they don't just use a debugger and always got unsatisfactory answers that seem to hide the fact they have a hard time using the tool effectively. Of course, like you say, using a debugger effectively in many settings is difficult. A multi-threaded program setting a breakpoint and inspecting state difficult, as finding the right thread to inspect can be difficult -- not to mention having other threads alter your debugger's display state by breaking once you're already inspecting the correct thread. I've had this happen with native software (e.g. C++), for which rr is great (but unfortunately I've worked mostly on Windows), but also for backend software written in higher level languages such as Python. Using a classic debugger effectively in a multithreaded environment requires being aware of the issues you're about to encounter, and knowing how to deal with them with the debugger's tooling -- which ranges between difficult and not possible. For example, with Windbg - it is possible to break on the correct thread, but setting a conditional breakpoint (even a simple one) is nothing less than an incantation to someone who's never used it before. As such, the cost of entry is very high, and I've watched kernel driver developers use prints and asserts to debug. While at my previous job, I thought that logs are a really crappy way of debugging an issue in production when it arises - if you have all the right information and can fetch it quickly, great. But often, that's not the case - you might end up needing to add logs, or worse - the issue will be in a hotspot where you simply can't log - since you'll have _too much_ data, or you can't log everything for compliance reasons. I had a similar idea to Rookout's (and even almost accidentally raised money just by talking to the right people, although I was not prepared at the time to start a company), and discovered Rookout later on, going on to use them. When I left my last job, I joined Rookout because -- like O'Callahan (and Pernosco) -- I'm very much a strong believer that the way debugging is today isn't good enough, and can be a lot better. Nowadays Rookout's product for interactive debugging backend apps is pretty much complete, works well and fast -- and we're exploring other ways to make debugging easier. Rookout's a bit different to rr/Pernosco in that it allows you to collect variables from anywhere during runtime (instead of allowing you to replay a recording) -- it's a bit similar to allowing you to add a log that logs locals() anywhere, except without changing your code, redeploying, etc.
• ensiferum 7y ago
  I find that the problem is that the better you become at engineering (i.e. designing and writing your code) the worse you become at debugging. This is because in a well designed system you have many (as many as possible) independent components that you can look at one piece at a time and you've written the appropriate unit tests for them. And subsequently you don't spend that much time in the debugger. In my experience this way most bugs (integration bugs aside) are rather trivial. In my own projects (over the past +15 years) most of my debugging involves just running gdb on my unit test until a test assertion fails, setting a breakpoint before the failed assertion and rerunning so that I can step through the computation. On another note there's actually this thing I'd like to call "debuggability". I find that if you just thinking of this as a measure of how easy your program is to debug and when making design decisions you think of "which design makes this thing easier to debug" your program will be simplified. To me mean high "debuggability" at design level means cohesive independent components which makes it easy to write unit tests for them which then subsequently makes it easy to debug them as well. Other thing is sometimes I see people write code like this: getFoo()->getBar()->doComputation(); or something like doComputation(getFoo()->GetSausage()); And depending on your debugger this can be super inconvenient to debug through. For example with gdb trying to step into any specific function basically requires you to find that function and set a breakpoint there. Otherwise you'll have to go through a lot of unrelevant stuff (especially if you're for example using std::unique_ptr) If you used some temporary variables to store the results of those function calls again you'll make your program more "debuggable".
  • int_19h 7y ago
    Any decent high-level debugger will provide some way to distinguish between your code and stdlib code when stepping, such that you don't end up inside std::unique_ptr and similar. Good ones will let you set a breakpoint on a subexpression, so in this case you'd just highlight the call that you want, and it'll stop right before it happens.
  • sfink 7y ago
    > For example with gdb trying to step into any specific function basically requires you to find that function and set a breakpoint there. step, then finish. Repeat until you're in the right subexpression. Step will enter a subexpression. If it's not the one you want, finish will run until it's done. Repeat.
• davidhyde 7y ago
  The Visual Studio .net (c# in particular) debugging experience has always been excellent in my opinion. One great plus for a managed language. I guess that the closer you get to the metal the more difficult it is for a debugger to do its magical things and hence the poor experience for c++ devs.
  • osullivj 7y ago
    Mixed langs on the same stack view in VS was a breakthrough for me, coding in a mix of C# and C++.
  • roca 7y ago
    Apart from Intellitrace, AFAICT it's a great implementation of the traditional stop-and-inspect debugger feature set, but that's all. Intellitrace adds something, but not full access to all program states like rr (or Pernosco). So I think your expectations could be higher :-).
    • GordonS 7y ago
      > it's a great implementation of the traditional stop-and-inspect debugger feature set, but that's all It provides more than that - the "Edit and Continue" feature is invaluable, allowing you to pause execution at a breakpoint and change the code, such that the new code runs when execution resumes. As an aside, Rider also has this feature, and I find it to be much faster and more reliable than VS. Visual Studio also recently got "Time Travel Debugging"[0], which lets you record execution then replay in the debugger. I haven't actually tried this yet. [0] https://devblogs.microsoft.com/visualstudio/introducing-time-travel-debugging-for-visual-studio-enterprise-2019/ https://devblogs.microsoft.com/visualstudio/introducing-time...
      • roca 7y ago
        Ah yes, TTD is a real game-changer. It compares fairly well with rr, though the recording overhead is often a lot higher. Lacks a lot of the Pernosco feature set though. I think that not many people have used TTD yet so when people talk about how great the VS debugger is, they're not really talking about TTD.
• alkonaut 7y ago
  I think some times the conversations about debuggers is one where developers talk past eachother because a developer who uses a high level language debugger in a fancy IDE to debug only high level code, vs. one that uses a native language debugger using a command line tool, will have wildly differnet experiences and expectations of what the tool can do, and especially about how discoverable and simple those features are. I belong in the first camp (Debugging to me means "running", It's not a tool I dig out to actually de-BUG something, it's just the tool I use to run my code every time).
  • clarry 7y ago
    They also talk past each other because the value of debuggers vastly depends on what you're debugging. It's relatively easy to get value out of a debugger in a straightforward synchronous userland process. It's much harder in a complex embedded application full of threads, callbacks, asynchronous execution, and a target where you may need to set up something like remote debugging and potentially tweak the whole system config to make it possible in the first place. This is also more likely to be the kind of system that doesn't run from your fancy IDE..
• The_rationalist 7y ago
  Is there something similar to RR for Java?
  • roca 7y ago
    Yes, I think Chronon is still available: http://chrononsystems.com/products/chronon-time-travelling-debugger http://chrononsystems.com/products/chronon-time-travelling-d...
  • yoannbuch 7y ago
    Not an actual debugger, but this tool can record Java applications (unit tests or full-blown applications) while they run, and then you can explore visually what happened through a web interface: http://findtheflow.io http://findtheflow.io. It's similar to rr because you can go back and forth in time, but unlike rr it doesn't record much of the state (besides return values) and instead focuses primarily on the execution flow by the threads across the different methods, classes and packages.
• rhinoceraptor 7y ago
  There's another class of debugging tools: tracers such as dtrace and bpftrace. There are a lot of bugs that you just can't reproduce in a debugger, or even in a development environment at all. You need to be able to debug in production safely and without huge performance penalties.
  • non-entity 7y ago
    I actually looked at learning dtrace recently, wondering if it could help me with a handful of isuses, but so far I've been unable to find information related to its capabilities, or up to date guides / books/ tutorials
    • rhinoceraptor 7y ago
      Brendan Gregg's website is a good resource, for both dtrace and ebpf: http://www.brendangregg.com/dtrace.html http://www.brendangregg.com/dtrace.html
• Crinus 7y ago
  > As far as I know, the only way to fix this situation is by building tools so much better than the free tools that the absurdity of refusing to pay for them is overwhelming, and expectations shift. This of course will only work for a little while until someone who has more time than money (or is a big company that wants to commoditize the tool) will build a command-line version of it on Linux. Weird UX and the need for a spaghetti of shell scripts to integrate with vim/emacs/vscode/sublime/ed will soon follow with an Eclipse addon that nobody will use a bit later. After a macOS port, assuming these are still a thing, Apple may create a nice looking UI and integrate it with Xcode or someone like Panic may create a good front end and sell it for ~$99. Most people will keep using Windows, think Visual Studio has the best debugger and everyone will be happy. Those that learn about the "overwhelmingly better approach" will consider the free one the best one and its spaghetti of shell scripts approach the obvious best approach, because they wont have any real deep experience with the paid tools (either the original one or the macOS shiny wrapper - which will be considered as unnecessary by most anyway) to properly judge.
  • roca 7y ago
    Yeah, nah. I sympathize with your point of view, but implementing Pernosco requires some serious science. If you try building a database of all memory and register changes in a naive way, it isn't going to work at all for nontrivial programs. Of course it could be cloned, but you would need a very good team and a lot of work. Also either you start with rr as a base, in which case you need people with deep rr knowledge, and I know their names, or you build or buy your own record and replay framework --- more time and money.
    • souprock 7y ago
      It's no longer likely at all, but some years ago my workplace nearly open sourced a serious tool. That would have been big news, with the sudden availability of source for a tool that does rr-like stuff. Some other company could do it. Remember that we've gotten Navigator (now Firefox), Blender, OpenSolaris, OpenOffice, and the .net stuff. Governments can surprise us as well; the USA did that with Ghidra. I'm not expecting it anytime soon, but surprises happen.
• henrik_w 7y ago
  I prefer good logging to debuggers. Often, you can't attach a debugger to the production system, but you can get logs. Also, you are often interested in the whole sequence of events leading up to the problem, not just the current state. And, to find where to look in the first place, you need to get an idea of what's wrong - a debugger doesn't help with that. https://henrikwarne.com/2014/01/01/finding-bugs-debugger-versus-logging/ https://henrikwarne.com/2014/01/01/finding-bugs-debugger-ver...
  • emsy 7y ago
    Those are orthogonal to each other and you should use both. But in development, a good debugger with data breakpoints, logging and conditional breakpoints is much more focused than an overly verbose log.
    • henrik_w 7y ago
      It's not overly verbose, and it's something you need anyway.
      • emsy 7y ago
        Overly verbose in the context of debugging. Logs almost can't be too verbose, because as you said, they should be able to tell you the sequence of events that lead to the error.
  • deleted 7y ago
    [deleted]
  • sfink 7y ago
    Logging always has a tradeoff between utility and overhead. It would be great to log everything, but it would impact performance too much. So you end up with logging levels, categories, release-only logs, etc. Pernosco has something for that too. Since it has replay capability, it can replay with more logging than the original execution had. So to some degree, you can have your cake and eat it too. It's not perfect: I think it requires some level of integration with a given logging system, it's not going to do the full I/O to send your logs to where they would ordinarily go (syslogd or whatever), and you can't log values with debug-only code that wasn't compiled into the binary. But it's still a large boost to the fundamental tension around logging.
• verdverm 7y ago
  Since I switched to Golang as a primary language, I have not needed a debugger. Prints, logs, and stack traces on exceptions has been sufficient.
• wheybags 7y ago
  I don't care much about fancy features. I just want a simple debugger GUI in my IDE that works consistently, and never crashes. Until I have that, I feel like pursuing even more functionality is pointless. (Context: c++ developer on Linux. The only debugging experience I've ever had that approaches what I describe was in c# on visual studio)
• nojvek 7y ago
  Author goes on a big rant and says rr solves everything. Big claim with little backing. There’s tons of languages. The closest I’ve seen to a sane debugging experience is vscode. It works the same across languages. Python, browser js, node js, go, Java etc Mostly because they all share same underlying debug adapter protocol. Now if you truly want to make debugging phenomenally better. Get those new apis in the protocol for reverse debugging and the likes. Then make it happen across a bunch of languages. I will pay It. I know most people will. We just want something uniform that works Log debugging is easiest. Why ? Because it’s simple AF. Running a 1000 things in parallel. Aggregate the logs and filter what you need. Going through layers of virtual computation? Physical -> Node pool -> service -> pod -> container. Write to log, aggregate. There’s been a lot of promise but haven’t seen anything that beats the simplicity of structured logs + good log analyzer tool.
• howard941 7y ago
  Sometimes just having a debugger is nice. There's an entire ecosystem based on cheap Arduino boards but the Arduino (teensy, anyway) intentionally hides its debug SWD pieces away to prevent pirating, leaving the developer high and dry with nothing but printf debugging.
• randomsearch 7y ago
  Just came here to say - JetBrains IDEs have fantastic debugging.
• PretzelFisch 7y ago
  Posts like this reminded me how spoiled I am now with the visual studio debugger for c#. I think it's still one of the best in the industry.
• deleted 7y ago
  [deleted]
• C1sc0cat 7y ago
  Try working in older languages with no real debugger that only has print statements
• pizlonator 7y ago
  This is such a great rant. One bit I’d add is the bad attitude towards debugging in optimizing compilers. It’s not obvious to me that optimizing for perf at the cost of debugging fidelity is a good trade off but here we are. Like, if you could save enough time debugging with better tools then you’d have more time to optimize. And maybe if compiler devs confronted full-speed debugging head on then we’d get to perf that is good enough for everyone and we’d all be more productive. Or maybe it all sucks and all we can do is bitch about it.
  • sfink 7y ago
    You can always tune it one way or another with the optimization level (-O0, -Og, -O3, ...) But I agree. That doesn't really help when you need to debug release binaries, which are understandably at maximum optimization levels. gcc is actually pretty good at preserving debuggability through optimizations. clang? Not so much.
  • roca 7y ago
    Glad you enjoyed it. For clang users things would get a lot better if someone just fixed a lot of clang bugs. Omniscient debugging creates new opportunities to overcome the impact of optimizations. E.g. are you trying to display the value of a variable after its live range ended? No problem, look back in time to when it was live. However, in many cases we need extensions to debuginfo to take advantage of that; e.g. DWARF can't express the difference between "variable outside live range, but has not been modified" and "we don't know where it is". If we had a debug build and an opt build of the same program then there are some mad-science approaches that would be fun to try. So I actually think the perf/debuggability tradeoff is not fundamental and with the right resources we could mostly make it go away.
    • pizlonator 7y ago
      JSC tries to go into the tricks you describe for the purpose of optimizing OSR exit. It’s really hard. There is a register pressure overhead from situations where the program thought a thing was live but really at that program point some of the necessary inputs to the value you’re trying to recover (and the value itself) would have been trashed by regalloc if it wasn’t for the need to be able to rematerialize that value. That translates into register pressure and you can feel it even on forgiving platforms like arm64. I believe that this overhead in conditions where the codegen is otherwise behaving well is around 5%. So not terrible, but that’s my call of the price relative to -O3 or whatever.
• garaetjjte 7y ago
  >my messages about Pernosco, which I think is much more interesting and important, have relatively little exposure It is not very surprising given that it is a SaaS which doesn't even list pricing on its page. >I want everyone to know about Pernosco, not just to attract the customers we need for sustainable debugging investment, but so that developers everywhere wake up to the awful state of debugging and rise up to demand an end to it. This is rather conflicting statement as Pernosco business model seems to be solely targeted at bigcorps paying big money, and not developers promoting Pernosco through using it in own hobby projects. (I don't know if this is actually true, but email-us-to-just-get-a-quote is huge discouragement)
  • mepiethree 7y ago
    It is also very difficult to reach a usage example or a workflow on their site. There is no demo. I agree that debuggers suck, but I don't see any clear/obvious evidence that Pernosco doesn't suck. I really just have to take their word for it. I don't want to pay an undisclosed amount of money for the opportunity to spend an unknown amount of time to get started with a new tool that may very well suck.
    • roca 7y ago
      There's a Youtube video right on the landing page: https://pernos.co https://pernos.co That video links to where you can test that session out for yourself: https://pernos.co/debug/e9UFXkGq__8KQB-N6n59lA/index.html https://pernos.co/debug/e9UFXkGq__8KQB-N6n59lA/index.html
  • deleted 7y ago
    [deleted]
  • roca 7y ago
    It's true that we aren't opening the gates for people everywhere to use Pernosco for their hobby projects, because we simply don't have the resources to support that. Hopefully that will change.
• toolslive 7y ago
  you used to have them: Borland had a good IDE with a good debugger for Pascal, C, ... (even Prolog) in the early 1990s. IBM's visual age (Smalltalk, Java) was great too: stop in the neigbourhood of the bug, edit the code, and continue without restarting... Now we're exporting logs from a docker in docker on travis. O tempora O mores
• timwaagh 7y ago
  Second time I saw an ad for this product on this site. Don't get me wrong. I like the idea of innovation in this space. But I think they should just buy advertising space.
• StillBored 7y ago
  The problem with runtime debuggers is that they are only really useful in a development environment. Outside of that you have to rely on core dumps, logging and various other post analysis tools. Which is why projects have to built the debugging into the product in one form or another. A topic that is far to broad/long for a posting here, but most projects end up implementing in various forms if they live long enough.
  • roca 7y ago
    With record-and-replay debugging that is not true. People can and are capturing failures outside the development environment (e.g. in CI) using rr and debugging those recordings in Pernosco, for example.
• Kapura 7y ago
  I use the Visual Studio debugger on an almost daily basis, and it'll be hard for me to use another tool if/when I leave my industry. Conditional and data breakpoints are gamechangers. To be frank, I didn't use debuggers much at all throughout my tenure at university, my internship at Amazon, and my first few gigs in the games industry. I still had bugs, but I was able to solve them mostly through printf, gut checks, and dousing rods. But when I got to AAA console game development, the leads had to show me how to use the immense power of the VS debugger, and now I have difficulty imaging life without it. Different scales of problems require different tooling to manage.
• seanmceligot 7y ago
  Break-point debuggers take too long in most environments (with a few exceptions). I use linters, compilers and type checkers as my first line of defense. Next, logging and unit tests. Lastly, I like to have in interactive command line REPL with good tab completion if one is available and not to difficult to setup. In python, for example I use: pycodestyle, pyflakes, mypy and logging, and bpython. For web apps, reload time is the most important factor. Work on reducing the time to compile, reload and test.
  • roca 7y ago
    But have you used a really good debugger, e.g. a record-and-replay debugger that lets you jump back in time to where some selected variable was modified? The whole point of the OP was that just about all debuggers suck, but debuggers can be a lot more powerful.
• gHosts 7y ago
  I once thought I could do better than gdb.... So I dug down into the bowels to see why it's so lame...... Oh my. The hideous complexity imposed by (multiple) CPU designs, OS interfaces, the vast mismatch between the code as written and the code as compiled. The complexity of the DWARF standard.... Sigh! Sorry gdb dev's... I'm sorry I ever thought bad of you guys... You guys are doing way better than I'd ever do.
• gnufx 7y ago
  Concerning making money with proprietary debuggers, DDT (no, not that one) and Totalview are very expensive parallel debuggers of longstanding. Totalview has some form of reversible debugging and is unusual in not using GDB. They're actually rarely used in my experience, for various reasons -- not just because they're un-affordable for most sites.