5 ms·
I will concede that rr is likely extremely useful, but as a counterpoint to this and the entire article, getting bugs earlier and earlier in the development cyc
by jchw 7y ago
I will concede that rr is likely extremely useful, but as a counterpoint to this and the entire article, getting bugs earlier and earlier in the development cycle beats better debugging any day. I will definitely try to add rr to my toolbelt if Ryzen support ever lands, but I still prefer catching problems by writing fast test cases instead. It's like debugging that happens automatically.
- olodus 7y agoI don't see these as separate solutions. Running your tests in a debugger usually makes it so much faster and easier to find where it went wrong. I would argue find bugs at compilation would be a separate solution from this and even earlier in the dev cycle.
- jchw 7y agoIndeed, finding bugs at compilation is better, which is why I prefer languages with strong typing. I reread my comment to make sure, but I am being painfully explicit that good debug tools are still useful. But I prefer good tests over even good debug tools. I very rarely even printf debug as a result of automated testing. This isn’t really just lip service; I have a project right now where I literally can’t attach a debugger or run locally. Of course, that’s a flaw and unreasonable. But, with good test hygiene, you can still be pretty productive in the face of this. Not only that, pushing code feels safe, even without the ability to actually run the code locally. Do I think debugging is bad? No. I think it’s worse than testing, which when done right can save you from a hell of a lot of manual debugging, even in the face of complex software with networked dependencies. I get ire on HN every single time I mention the virtues of testing instead of debugging. I suspect people literally don’t believe you can exchange quantities of one for the other, but where I’m standing it’s obviously possible. Same thing comes up with strong typing frequently, where you have people making the claim that typings don’t prevent bugs - but anyone who’s transitioned a medium sized code base to TypeScript can attest to the improvements it can bring. In my opinion, automated tests are almost literally just debugging done by a script instead of a human. edit: It's also being pointed out to me that rr is being used to debug failing tests, which is not something I thought of and my ignorance may be contributing to some of the reason my comment was so poorly received here rather than the testing vs debugging comparison. (In many cases, for unit tests, I had never thought of using a debugger to try to find the problem, because the actual problem is often made obvious enough by the failed assertion output.)
- de_watcher 7y agoYes, automated tests are almost debugging by a script. If you've ever used GDB in command-line mode then it's evident to you that it's the same thing. You can write tests of any level. With a bit less funky GDB it's completely viable to solve a hard task of writing unit tests for the old untestable code without refactoring for example.
- sfink 7y agoI agree on the value of tests, but it somewhat depends on what you're working on as to how hard it is to get a certain level of test coverage. I tend to work on a garbage collector, which is notoriously good at collecting the flaws in all memory manipulation code anywhere, and funneling them down to a couple of crash sites where we are traversing memory. We use assertions very heavily to get mileage out of everyone else's test cases, and we add regression tests studiously, but we still end up in the debugger a lot. Also, one surprisingly useful feature of rr is that just about everything is debuggable without needing to plumb through debugger invocations through your whole system. It will record an entire process tree, so you can go back after the fact and select which process to replay debug. It's incredibly useful. I've been so happy to ditch my weird wrapper scripts to run cc1plus under a debugger while compiling with a buggy static analysis plugin I'm working on. Now I just run `rr record make` and wait for it to crash. It's magical.
- roca 7y ago"a project right now where I literally can’t attach a debugger or run locally" ... this is exactly where record-and-replay tools like rr can change the game. You can (potentially) run that code under rr on the remote machine --- without interrupting it, which is often problematic --- then upload the recording somewhere (e.g. Pernosco) for debugging.
- roca 7y agoOP here. I am obsessed with improving debugging, but I am also obsessed with Rust. Pernosco is mostly written in Rust. I am a huge fan of strong typing and tests, but I know from experience that they don't eliminate the need for debugging in complex systems.
- 7y ago
- de_watcher 7y agorr is exactly the harness to debug tests. We have a system when a test fails you get a rr replay of it to debug. It's the primary use for it.
- jchw 7y agoOK, I'll just be honest: I had no idea that's how people used rr. I saw demos some years ago and assumed one used it similarly to gdb but with rewind. I think this is an awesome use case for rewind debugging, that I had never thought of, and look forward to hopefully seeing it run on AMD Ryzen processors in the future, since I've bought into that ecosystem too much to go back already.
- pmoriarty 7y ago"I saw demos some years ago and assumed one used it similarly to gdb but with rewind." You can do reverse debugging in gdb. Here's how: Run: gdb --args /usr/bin/foo --bar baz Type "start", then "record", then "continue" Now can try a number of reverse-debugging commands, for example: * reverse-stepi * reverse-next * etc
- roca 7y agogdb's built-in recording doesn't scale. You can't record across system calls, thread context switches, etc, and it's about a 1000x slowdown during record and replay. (rr is more like 2x.) It is, unfortunately, an attractive nuisance, because a lot of people have tried it out and concluded that record-and-replay debugging is useless.
- roca 7y agoOoh, have you blogged about that anywhere?
- hsivonen 7y agoIt's nice to say that catching bugs earlier is better. But what if that time is long gone? Lately, I've used Pernosco to debug my changes to a multi-process-relevant decade-old part of two-decade-old codebase, which was single-process when written, only later became multi-process-relevant, and recently became even multier-process. (And there are test cases, but there's a huge mass of them and they depend on decade-old comments not actually being true the interesting situations.) Pernosco is very useful for debugging multi-process interaction in cases that would be very hard to debug even with rr.