8 ms·

Those of us who have compute-bound workloads (scientific simulations in my case, but this often comes up in gamedev and elsewhere) often complain about how unus

Those of us who have compute-bound workloads (scientific simulations in my case, but this often comes up in gamedev and elsewhere) often complain about how unusably slow debug builds impede our workflow in C++. One fairly common solution to this in extreme cases is sticking to a mostly C-like subset of the language or just using C. I don't have much experience with Rust, but judging by the results here, debug builds in Rust are effectively useless, even compared to C++. How do Rust programmers deal with this? Do you just give up and resort to printf debugging, or are there ways to improve the situation?

kibwen 8y ago
This is going to sound unbelievable, but I don't really ever do runtime debugging of my Rust code. Nearly all my debugging happens at compile-time (which is faster to iterate with thanks to cargo-check skipping codegen and linking). That said, I do know some people with large Rust codebases who do value the ability to run debug builds, and the slowness of -O0 binaries is a sore spot for them. As with C, you can try -O1 for a compromise between runtime and compiletime.
- jordigh 8y ago
  The issue is that the optimised code is not the same as the unoptimised code, and debugging optimised code sucks. It appears to the debugger that your execution is jumping around erratically and often variables don't exist because the optimiser decided that they didn't need a reason to exist. It's a problem with any kind of compiled language that runs through an optimiser, whether C, C++, Rust, or D. Maybe that's why so many people just don't use debuggers, although my philosophy is that everyone should use a debugger, but maybe they just haven't found the right one yet.
  - loeg 8y ago
    It doesn't have to suck nearly as badly as it does today with LLVM-based toolchains, including Rust's. With C and C++, GCC does a much better job of maintaining accurate DWARF information with performance at least as good as Clang. (And optimized code certainly shouldn't be "jumping around erratically," simply because that results in poor performance on modern CPUs with multi-cycle instruction pipelines and imperfect branch predictors.) A variable doesn't have to be in a single register throughout an entire function, or even present in registers / memory at all! DWARF is capable of representing storage locations that change over the course of a subroutine, as well as synthesizing values from other registers/memory, simple operations, and constants. Compilers can do a lot better (although certainly not 100%) than they do today without sacrificing any of the optimizations they make, and especially Clang/LLVM have a lot of room for improvement. I highly recommend this blog post on the subject: https://backtrace.io/blog/engineering/compile-once-debug-twice-picking-a-compiler-for-debuggability-1of3/ https://backtrace.io/blog/engineering/compile-once-debug-twi...
- marksomnian 8y ago
  How do you debug logic errors in that case? I imagine that most logic errors can't be caught by the compiler, although I could be wrong.
  - steveklabnik 8y ago
    I come from Ruby, where debuggers are useful but also not often used. I find that I tend to write more and smaller unit tests than people who reach for debuggers. I debug logic errors by using println or writing more tests. And yes, Rust is still very susceptible to logic errors.
    - pjmlp 8y ago
      Developing with powerfull debuggers is an experience that kind of resembles using a REPL, specially with languages that enjoy fast compiles. I advise devs to go through Xerox PARC and ETHZ papers about interactive development, or Apple's Object Pascal/MCL/Dylan/Hypercard, on was the birth of IDEs.
      - steveklabnik 8y ago
        Fun fact: I loved coding in HyperCard as a kid.
  - staticassertion 8y ago
    Most logic errors can definitely be caught by the compiler if you're taking the approach of encoding your logic into your type system. I've done this with real codebases (in Python as well, actually, using Mypy) and it's been very effective.
    - marksomnian 8y ago
      Can it catch something like accidentally writing "x == y", when you meant to write "x != y"? I feel like this kind of bug can be insidious enough to waste lots of debugging time, yet difficult to catch until you run the code.
- WalterBright 8y ago
  As my decades of programming experience accumulate, the source of most of my programming bugs shifted. Earlier it was dumb mistakes, memory management errors, seg faults, etc. Now it's logic errors due to design mistakes and failing to understand properly what needed to be done. A fair bit of the D programming language design is there to head off the detail mistakes people make. A more subtle thing in the language design is to make the designs less prone to error.
burntsushi 8y ago
In my work (text search), if I'm debugging, then the vast majority of my time is spent debugging performance problems. In those cases, it only makes sense to debug with all optimizations turned on. If I'm debugging logic problems, which is a bit more rare, then I've found debug builds to be fast enough in most cases. When they aren't, then I turn on release mode. Incremental compilation mostly makes compile times bearable, but not always. I suspect the experience here will differ quite a bit depending on what you're working on and what your preferred workflow looks like. > Do you just give up and resort to printf debugging FWIW, I've been printf debugging since I started programming, regardless of the language I use. To be fair, if I spent more time in a predominantly memory unsafe language, then I'd probably use a debugger more.
- WalterBright 8y ago
  I mostly use printf. A debugger is only good for telling you where it seg faulted and the stack trace. The reason is because I write a custom printf to print exactly what I need to know. Debuggers just bury you in irrelevant output.
  - yc-kraln 8y ago
    A good debugger is good for quite a bit more than that. Watches, break points, performance/profiling. I actually can't believe you're hating on debuggers.
    - SlowRobotAhead 8y ago
      It's funny to me as an embedded programmer seeing people write about how they prefer printf to actual debugging. My printf command can take MANY TIMES longer to run than most of the code that I'm trying to fix. Maybe it's a software vs hardware thing, but I would end it all if I had to work hardware without breakpoints, watches, and step-through.
      - burntsushi 8y ago
        Why is it funny? And why do you not consider printf to be "actual" debugging? I mean, if printf wasn't available to me or was for some reason otherwise inconvenient, then I would look for other avenues to debug, perhaps by using a debugger! This isn't that mystifying. As I said above, this is very heavily dependent on preferred workflows and what you're working on. Long ago, I remember doing some robotics work in C, and a debugger was invaluable.
staticassertion 8y ago
Can't speak to others, but it's funny to think of "printf debugging" as a last resort when it's easily my go-to. I'll often add some println's and re-run tests to trace through what happened. I only step into a debugger when I'm really wtf'd, and I don't think that's happened to me even once with Rust (it used to happen a lot with C++, since printf debugging mixes so poorly with segfaults etc). No question that long compile times are a pain though. I generally use 'cargo check' 99.99% of the time. Thankfully there's work going into making debug builds faster by, if I recall, replacing the llvm backend with some other thing I don't recall the name of.
- Avi-D-coder 8y ago
  The code generator that may complement/replace llvm for this use case is Cranelift. https://github.com/CraneStation/cranelift https://github.com/CraneStation/cranelift
floofy222 8y ago
> How do Rust programmers deal with this? You compile with -C opt-level=1.
- v_lisivka 8y ago
  Cargo.toml: [profile.dev] #opt-level = "s" opt-level = 2
berkut 8y ago
Not talking about Rust here, so not really an answer, but in the film VFX industry (rendering in my case) pretty much all the high performance code is C++ or CUDA, and before we moved to c++11, I'd worked out that for STL iterators, at least in the 4.2/4.4 GCC versions, pre-caching the end() iterator of an STL collection before the loop instead of calling it each time as a control condition of the loop made faster debug runtime builds, so I got into the habit of doing that. In the optimised (O2/O3) builds the compiler effectively did that optimisation anyway, but not at O0. It meant loop code in the source was longer, but made measurably (~10% if I remember correctly) faster debug builds on code which did a lot of vector iterating. I have also noticed that often some of debug builds' slowness is actually due to asserts being on in debug builds as opposed to optimised builds, at least in some of the code I work on (which in some cases makes very heavy use of them) - removing asserts can in some cases bring things back to debug builds being ~5x slower than optimised compared to ~30x which is more acceptable.
- rwbt 8y ago
  Thanks for the tip. I too work in rendering and often wondered if pre-caching end() would make significant difference but figured its best left to the compiler.
- renox 8y ago
  Have you tried -Og? I would try this before modifying the sources..
m0zg 8y ago
Another fix to build times is to use reliable fully incremental builds, such as implemented by e.g. Google Bazel. Bazel can also cache intermediate build artifacts to speed up the builds even further.

kibwen 8y ago

This is going to sound unbelievable, but I don't really ever do runtime debugging of my Rust code. Nearly all my debugging happens at compile-time (which is faster to iterate with thanks to cargo-check skipping codegen and linking). That said, I do know some people with large Rust codebases who do value the ability to run debug builds, and the slowness of -O0 binaries is a sore spot for them. As with C, you can try -O1 for a compromise between runtime and compiletime.

jordigh 8y ago
The issue is that the optimised code is not the same as the unoptimised code, and debugging optimised code sucks. It appears to the debugger that your execution is jumping around erratically and often variables don't exist because the optimiser decided that they didn't need a reason to exist. It's a problem with any kind of compiled language that runs through an optimiser, whether C, C++, Rust, or D. Maybe that's why so many people just don't use debuggers, although my philosophy is that everyone should use a debugger, but maybe they just haven't found the right one yet.
- loeg 8y ago
  It doesn't have to suck nearly as badly as it does today with LLVM-based toolchains, including Rust's. With C and C++, GCC does a much better job of maintaining accurate DWARF information with performance at least as good as Clang. (And optimized code certainly shouldn't be "jumping around erratically," simply because that results in poor performance on modern CPUs with multi-cycle instruction pipelines and imperfect branch predictors.) A variable doesn't have to be in a single register throughout an entire function, or even present in registers / memory at all! DWARF is capable of representing storage locations that change over the course of a subroutine, as well as synthesizing values from other registers/memory, simple operations, and constants. Compilers can do a lot better (although certainly not 100%) than they do today without sacrificing any of the optimizations they make, and especially Clang/LLVM have a lot of room for improvement. I highly recommend this blog post on the subject: https://backtrace.io/blog/engineering/compile-once-debug-twice-picking-a-compiler-for-debuggability-1of3/ https://backtrace.io/blog/engineering/compile-once-debug-twi...
marksomnian 8y ago
How do you debug logic errors in that case? I imagine that most logic errors can't be caught by the compiler, although I could be wrong.
- steveklabnik 8y ago
  I come from Ruby, where debuggers are useful but also not often used. I find that I tend to write more and smaller unit tests than people who reach for debuggers. I debug logic errors by using println or writing more tests. And yes, Rust is still very susceptible to logic errors.
  - pjmlp 8y ago
    Developing with powerfull debuggers is an experience that kind of resembles using a REPL, specially with languages that enjoy fast compiles. I advise devs to go through Xerox PARC and ETHZ papers about interactive development, or Apple's Object Pascal/MCL/Dylan/Hypercard, on was the birth of IDEs.
    - steveklabnik 8y ago
      Fun fact: I loved coding in HyperCard as a kid.
- staticassertion 8y ago
  Most logic errors can definitely be caught by the compiler if you're taking the approach of encoding your logic into your type system. I've done this with real codebases (in Python as well, actually, using Mypy) and it's been very effective.
  - marksomnian 8y ago
    Can it catch something like accidentally writing "x == y", when you meant to write "x != y"? I feel like this kind of bug can be insidious enough to waste lots of debugging time, yet difficult to catch until you run the code.
WalterBright 8y ago
As my decades of programming experience accumulate, the source of most of my programming bugs shifted. Earlier it was dumb mistakes, memory management errors, seg faults, etc. Now it's logic errors due to design mistakes and failing to understand properly what needed to be done. A fair bit of the D programming language design is there to head off the detail mistakes people make. A more subtle thing in the language design is to make the designs less prone to error.

burntsushi 8y ago

In my work (text search), if I'm debugging, then the vast majority of my time is spent debugging performance problems. In those cases, it only makes sense to debug with all optimizations turned on. If I'm debugging logic problems, which is a bit more rare, then I've found debug builds to be fast enough in most cases. When they aren't, then I turn on release mode. Incremental compilation mostly makes compile times bearable, but not always. I suspect the experience here will differ quite a bit depending on what you're working on and what your preferred workflow looks like. > Do you just give up and resort to printf debugging FWIW, I've been printf debugging since I started programming, regardless of the language I use. To be fair, if I spent more time in a predominantly memory unsafe language, then I'd probably use a debugger more.

WalterBright 8y ago
I mostly use printf. A debugger is only good for telling you where it seg faulted and the stack trace. The reason is because I write a custom printf to print exactly what I need to know. Debuggers just bury you in irrelevant output.
- yc-kraln 8y ago
  A good debugger is good for quite a bit more than that. Watches, break points, performance/profiling. I actually can't believe you're hating on debuggers.
  - SlowRobotAhead 8y ago
    It's funny to me as an embedded programmer seeing people write about how they prefer printf to actual debugging. My printf command can take MANY TIMES longer to run than most of the code that I'm trying to fix. Maybe it's a software vs hardware thing, but I would end it all if I had to work hardware without breakpoints, watches, and step-through.
    - burntsushi 8y ago
      Why is it funny? And why do you not consider printf to be "actual" debugging? I mean, if printf wasn't available to me or was for some reason otherwise inconvenient, then I would look for other avenues to debug, perhaps by using a debugger! This isn't that mystifying. As I said above, this is very heavily dependent on preferred workflows and what you're working on. Long ago, I remember doing some robotics work in C, and a debugger was invaluable.

staticassertion 8y ago

Can't speak to others, but it's funny to think of "printf debugging" as a last resort when it's easily my go-to. I'll often add some println's and re-run tests to trace through what happened. I only step into a debugger when I'm really wtf'd, and I don't think that's happened to me even once with Rust (it used to happen a lot with C++, since printf debugging mixes so poorly with segfaults etc). No question that long compile times are a pain though. I generally use 'cargo check' 99.99% of the time. Thankfully there's work going into making debug builds faster by, if I recall, replacing the llvm backend with some other thing I don't recall the name of.

Avi-D-coder 8y ago
The code generator that may complement/replace llvm for this use case is Cranelift. https://github.com/CraneStation/cranelift https://github.com/CraneStation/cranelift

floofy222 8y ago

> How do Rust programmers deal with this? You compile with -C opt-level=1.

v_lisivka 8y ago
Cargo.toml: [profile.dev] #opt-level = "s" opt-level = 2

berkut 8y ago

Not talking about Rust here, so not really an answer, but in the film VFX industry (rendering in my case) pretty much all the high performance code is C++ or CUDA, and before we moved to c++11, I'd worked out that for STL iterators, at least in the 4.2/4.4 GCC versions, pre-caching the end() iterator of an STL collection before the loop instead of calling it each time as a control condition of the loop made faster debug runtime builds, so I got into the habit of doing that. In the optimised (O2/O3) builds the compiler effectively did that optimisation anyway, but not at O0. It meant loop code in the source was longer, but made measurably (~10% if I remember correctly) faster debug builds on code which did a lot of vector iterating. I have also noticed that often some of debug builds' slowness is actually due to asserts being on in debug builds as opposed to optimised builds, at least in some of the code I work on (which in some cases makes very heavy use of them) - removing asserts can in some cases bring things back to debug builds being ~5x slower than optimised compared to ~30x which is more acceptable.

rwbt 8y ago
Thanks for the tip. I too work in rendering and often wondered if pre-caching end() would make significant difference but figured its best left to the compiler.
renox 8y ago
Have you tried -Og? I would try this before modifying the sources..

m0zg 8y ago

Another fix to build times is to use reliable fully incremental builds, such as implemented by e.g. Google Bazel. Bazel can also cache intermediate build artifacts to speed up the builds even further.