5 ms·

Eigen is the standard choice (for good reason!) for many linear algebra projects, especially in robotics, but there is a big downside users should be aware of b

Eigen is the standard choice (for good reason!) for many linear algebra projects, especially in robotics, but there is a big downside users should be aware of before they chose it. Eigen makes extensive use of expression templates in C++ to collapse complex operation sequences into streamlined and minimal calculations. This is generally ok, until you need a debug build. I've regularly seen debug builds of software using Eigen run 1000x to 10000x slower than the release build, which seriously complicates various debugging workflows. It also makes it a nightmare to run your test suite through valgrind, for example. I've seen several engineers attempt (and fail) to try creating/linking a release build of Eigen with a debug build of the rest of the program to try to regain most of that speed while still allowing a decent amount of debugability, but this is really hard due to all the aggressive inlining and heavy use of templates. In my experience, I would happily accept a 2x or more slowdown in linear algebra performance in release builds in exchange for significant boost in debug execution speed. If you're starting a greenfield project, you should consider how important decent debug performance is before choosing Eigen by default.

hasmanean 5y ago
This is a problem with C++ in general. In debug mode it has no notion of performance whatsoever.
gugagore 5y ago
I appreciate being made aware of this downside! Why does this happen with C++ debugging?
- hermitdev 5y ago
  In general, in template heavy libraries, what kills performance in debug builds is lack of inlining with optimizations turned off. And templates traditionally need to be inlinable by having their full definition in a header, so one can't even say build with optimizations on with debug info in isolation, because it has to be done where the templates are used, not in the library where they're defined (because there is no separately built library for the template). In other words, it's a real pain to get just the templates optimized while the rest of your code is not optimized for easier debugging.
duped 5y ago
You can create custom build configurations to set the optimizations and add debug info. Or just add instrumentation code (print debugging) which is more useful for debugging heavy numerical stuff most of the time anyway. Running anything through valgrind or cachegrind will have several orders of magnitude slowdown - that's inherent to how the tools work. To your last point, just add -g to your compile flags and see how far you get.
- bobsomers 5y ago
  I don't think you quite understand the problem. The problem wasn't turning on optimizations or getting debug info, the problem was getting Eigen-based linear algebra code to run at a sufficiently reasonable speed when optimizations were disabled because doing so would make other surrounding logic easier to debug (or you needed to run some analysis tool like Valgrind on a debug build). For example, we had simulation test suites which would run ~30 seconds of real-world time. If we can run those sims at 10x real time, each only takes 3 seconds, great! But with optimizations turned off, heavy Eigen code would run 1000x slower, which means the sims now take nearly 3000 seconds to run when bug hunting in the non-Eigen related surrounding logic. This already makes the test suite virtually un-runnable in debug builds, but if you want to then run those binaries through Valgrind (which tacks on another 100-1000x performance tax) you're now talking about something which is completely intractable. You can't run optimized builds through Valgrind without potentially introducing false positives, so you can't even run Valgrind on only the optimized builds... you just can't run it at all because your linear algebra library is slowing everything down. The core problem is Eigen's heavy use of expression templates, which the optimizer cuts through just fine. But without the optimizer turned on, it adds layers and layers and layers of unnecessary abstraction. Because these are all templates, C++ expands all this code at every call site, making it substantially less efficient and impossible to shim out by linking with an optimized build of Eigen itself. The only approach I've seen get close to making this work was an attempt by an engineer to use explicit template instantiation to create a separate binary with all the necessary Eigen functions which could be compiled separately with the optimizer turned on. However, this proved too nightmarish because since Eigen can template on the dimensions of everything, you need to know up front exactly all the various instantiations of Eigen types the program uses. This might be doable by writing a clang plugin or something, but doing it by hand was just way too much work because the codebase was huge, so we couldn't justify spending more time on it.

hasmanean 5y ago

This is a problem with C++ in general. In debug mode it has no notion of performance whatsoever.

gugagore 5y ago

I appreciate being made aware of this downside! Why does this happen with C++ debugging?

hermitdev 5y ago
In general, in template heavy libraries, what kills performance in debug builds is lack of inlining with optimizations turned off. And templates traditionally need to be inlinable by having their full definition in a header, so one can't even say build with optimizations on with debug info in isolation, because it has to be done where the templates are used, not in the library where they're defined (because there is no separately built library for the template). In other words, it's a real pain to get just the templates optimized while the rest of your code is not optimized for easier debugging.

duped 5y ago

You can create custom build configurations to set the optimizations and add debug info. Or just add instrumentation code (print debugging) which is more useful for debugging heavy numerical stuff most of the time anyway. Running anything through valgrind or cachegrind will have several orders of magnitude slowdown - that's inherent to how the tools work. To your last point, just add -g to your compile flags and see how far you get.

bobsomers 5y ago
I don't think you quite understand the problem. The problem wasn't turning on optimizations or getting debug info, the problem was getting Eigen-based linear algebra code to run at a sufficiently reasonable speed when optimizations were disabled because doing so would make other surrounding logic easier to debug (or you needed to run some analysis tool like Valgrind on a debug build). For example, we had simulation test suites which would run ~30 seconds of real-world time. If we can run those sims at 10x real time, each only takes 3 seconds, great! But with optimizations turned off, heavy Eigen code would run 1000x slower, which means the sims now take nearly 3000 seconds to run when bug hunting in the non-Eigen related surrounding logic. This already makes the test suite virtually un-runnable in debug builds, but if you want to then run those binaries through Valgrind (which tacks on another 100-1000x performance tax) you're now talking about something which is completely intractable. You can't run optimized builds through Valgrind without potentially introducing false positives, so you can't even run Valgrind on only the optimized builds... you just can't run it at all because your linear algebra library is slowing everything down. The core problem is Eigen's heavy use of expression templates, which the optimizer cuts through just fine. But without the optimizer turned on, it adds layers and layers and layers of unnecessary abstraction. Because these are all templates, C++ expands all this code at every call site, making it substantially less efficient and impossible to shim out by linking with an optimized build of Eigen itself. The only approach I've seen get close to making this work was an attempt by an engineer to use explicit template instantiation to create a separate binary with all the necessary Eigen functions which could be compiled separately with the optimizer turned on. However, this proved too nightmarish because since Eigen can template on the dimensions of everything, you need to know up front exactly all the various instantiations of Eigen types the program uses. This might be doable by writing a clang plugin or something, but doing it by hand was just way too much work because the codebase was huge, so we couldn't justify spending more time on it.