10 ms·
Eigen: A C++ template library for linear algebra
- conformist 5y agoEigen is a great library. A similar one to consider that can at times be slightly easier to use coming from a python background is armadillo: http://arma.sourceforge.net/ http://arma.sourceforge.net/
- yarky 5y agoI really like armadillo's syntax, it feels like your doing R/Matlab.
- structural 5y agoThat's not a positive for everyone, Matlab's API choices are a bit polarizing/confusing for people who don't use it.
- bayindirh 5y agoI’m using Eigen extensively in a project of mine. It’s extremely fast, versatile, easy to use and reliable. I’m a fan, one might say.
- jabl 5y agoSlightly tooting my own horn here; In case anyone is interested in a (trivial) comparison between them I have a tiny example project implemented both with Eigen and Arma (And Fortran and Python/numpy and Julia, FWIW): https://gitlab.com/jabl/tb https://gitlab.com/jabl/tb
- hcrisp 5y agoI wonder how Eigen compares to xtensor, which was inspired by Numpy and has support for views, slicing, and broadcasting? https://github.com/xtensor-stack/xtensor https://github.com/xtensor-stack/xtensor
- structural 5y agoEigen is similar to xtensor, but with 1) Eigen being significantly more mature, and used in production in simulation/research environments and 2) Eigen having not just basic operations on arrays, but also a full suite of linear algebra operations comparable to a full BLAS library, including a full test suite. xtensor wins on having better integration with Python and other languages out of the box. There's also some performance limitations due to being inspired by numpy, particularly with respect to memory management.
- patrick451 5y agoIt's not quite out of the box, but pybind11 has really nice support for eigen matrices and vectors and automatically converts them to/from numpy arrays.
- criddell 5y agoThe first thing I wondered was about the license. Fortunately, it mostly uses the MPL license: https://eigen.tuxfamily.org/index.php?title=Main_Page#License https://eigen.tuxfamily.org/index.php?title=Main_Page#Licens... > Note that currently, a few features rely on third-party code licensed under the LGPL: constrained_cg. Such features can be explicitly disabled by compiling with the EIGEN_MPL2_ONLY preprocessor symbol defined. Furthermore, Eigen provides interface classes for various third-party libraries (usually recognizable by the <Eigen/*Support> header name). Of course you have to mind the license of the so-included library when using them. > Virtually any software may use Eigen. For example, closed-source software may use Eigen without having to disclose its own source code. Many proprietary and closed-source software projects are using Eigen right now, as well as many BSD-licensed projects.
- rough-sea 5y agoThis library is used heavily inside Tensorflow. It is “production ready”.
- ska 5y agoFWIW it's been used in production long before tensorflow existed.
- RavlaAlvar 5y agoI can’t stress enough how this library is a saving grace. It would be incredibly difficult to port my python prototype code with numpy to C++ production without using Eigen.
- Const-me 5y agoUsing it for years, and mostly happy with that library. The performance is awesome for very long vectors / large matrices. It’s less than ideal for small things when the size is known at compile-time. If one knows SIMD intrinsics, in some of these cases the Eigen’s implementation can be outperformed by a large factor like 2-4. Also it’s very hard to mess with RAM layout of some things (like sparse matrices), just too many layers of abstraction and too much template metaprogramming. But still, out of the box the usability of Eigen is awesome. And until the code is written, debugged, integrated and benchmarked, it’s generally impossible to tell whether a particular algorithm gonna be a performance bottleneck. That’s why I’m mostly happy with the library.
- markisus 5y agoCan confirm the poor performance on small matrices (less than 20x20). This is a problem particularly for robotics applications when your entities are positions, velocities, etc. https://stackoverflow.com/questions/58071344/is-eigen-slow-at-multiplying-small-matrices https://stackoverflow.com/questions/58071344/is-eigen-slow-a...
- justshowpost 5y agoNo estoy de acuerdo con tus politicas
- szundi 5y agoAn other answer just debates wether that was a benchmarking issue
- markisus 5y agoMultiple other users confirmed the results and that one answer's author never followed up.
- Const-me 5y ago> when your entities are positions, velocities, etc. For use cases where FP32 precision is enough, I usually use DirectXMath library https://github.com/Microsoft/DirectXMath https://github.com/Microsoft/DirectXMath for that. That thing is cross-platform in practice. Even when building things for ARM Linux, it’s easy to copy-paste required pieces, NEON support is there. When I need FP64 precision on PCs, I usually proceed without libraries, using AVX intrinsics.
- Bancakes 5y agoBLAS vs Eigen Go
- gnufx 5y agohttps://github.com/flame/blis/blob/master/docs/Performance.md https://github.com/flame/blis/blob/master/docs/Performance.m... but ignore the SKX results with the old OpenBLAS there.
- onedognight 5y agoEigen calls BLAS when appropriate and with no overhead and the code looks close to the math. Eigen also supports small fixed size matrices where BLAS is not appropriate.
- ska 5y ago> Go Category error. One is a standard the other a library. BLAS covers a subset of what Eigen does, and Eigen can use BLAS/LAPACK routines directly from other libraries (e.g. MKL) for those things.
- bee_rider 5y agoI believe (haven't looked at benchmarks lately) Eigen has trouble beating a good BLAS implementation like MKL or BLIS at doing BLAS stuff, but it is more expressive and has the ability to do lazy evaluation/fuse operations. Anyway since you can get it to call your favorite BLAS/LAPACK library, these are really complementary projects.
- phkahler 5y agoMy only complaint is that using OpenMP Eigen can be slower with SMT than without SMT. They even suggest telling to use half as many threads as you have "cores" when "cores" means twice as many due to SMT. Otherwise, we've seen a 8-10x performance increase in SolveSpace (CAD) in some situations after switching from home-grown matrix operations to Eigen.
- stingraycharles 5y agoI think this is generally true if your workload is SIMD/AVX heavy: these types of “heavy” instructions cannot execute on a single core simultaneously.
- jhartmann 5y agoThis has been true for years, Intel CPU's can't efficiently perform math operations when hyperthreading is involved. Generally there is only a single shared FPU/AVX/SSE unit doing the math over two hyperthreads. Since the Eigen implementation often can keep that unit 100% busy, it makes no sense to try and run two threads at full tilt through the units. I tested all this very heavily before Eigen had AVX-512 support. In that environment there might be some differences and I would suggest you benchmark both configurations.
- Sesse__ 5y ago> Generally there is only a single shared FPU/AVX/SSE unit doing the math over two hyperthreads. Hyperthreading does in general share units (both ALU and others); that's what hyperthreading is. Apart from that, it really depends on what operations you're doing; e.g., modern Intel CPUs have three ports that can issue a 256-bit FMA, each, every cycle.
- phkahler 5y ago>> Hyperthreading does in general share units (both ALU and others); that's what hyperthreading is. Yes, I think the issue with Eigen is cache related. They apparently have optimizations that are aware of cache architecture and running 2 threads that share the same cache will screw that up, resulting in more misses. If this is the case, I'd prefer algorithms that are cache line size agnostic. It is still much faster than the simple hand-written code we had before!
- digiou 5y agoGod bless Eigen, I am using it to implement the state matrices of a Kalman Filter and it's a joy to use its APIs.
- Rayhem 5y agoEigen has one of if not the best linear algebra APIs I've ever seen. In particular, vectors are column vectors by default and you never need to touch row vectors (if I can editorialize, row vectors shouldn't exist at all), and vectors are not simply "n-by-1" matrices -- they're true vectors.
- mereel 5y agoCould you elaborate a bit more about the difference between Nx1 matrices and "true vectors"?
- the_svd_doctor 5y agoOne has shape (N, 1) and one has shape (N). Always having a trailing 1 is a pain in many settings. Matlab has that “trailing 1” all the time and it’s a PITA.
- patrick451 5y agoWhy do you say it's a pain? After working with numpy for the last two years, that's the part I miss most about matlab.
- Rayhem 5y agoThere are many reasons the "-by-1" doesn't make sense. * If a vector is an n-by-1 array (a "matrix"), why not an n-by-1-by-1-by-1-by-1? * What is a row vector in something like a function space? Is sin(t) a row or column vector? * You _can_ think of a matrix as acting "to the right (on a column vector)" and "to the left (on a row vector)", but this disrupts the notion of a matrix as a linear function acting on a thing (since it can act on two things). It's much more consistent to define a matrix acting on a vector, m(x), independent of "direction" and then define the transformations that let you go in the opposite direction (take the transpose of m which is a whole new matrix). * Are physical quantities like velocity a row or column vector? Convention dictates they're column vectors, so in what scenario would it become a row vector? If I have a velocity, v1, that's a row vector and another, v2, that's a column vector, is it appropriate to calculate the projection of v1 onto v2? Should the projection operation be "aware" of the orientations? If all vectors are the same ("column vectors"), this isn't a problem -- you can always define the projection operation without special cases. * In matlab specifically, if you have an n-by-1 array `a` and a 1-by-n array `b` with the same elements, then `a(i) == b(i)` but `a` and `b` are not the same thing.
- madduci 5y agoA great library, a cornerstone. Many big libraries build on top of it (e.g. OpenCV, PointCloud Library)
- asimeqi 5y agoCan you elaborate on how OpenCV is built on top of Eigen? From what I can google it seems that OpenCV can interoperate with Eigen but is not build on top of it.
- madduci 5y agoThere are functions related to camera calibration and adjustment of images whose internals are built using Eigen Vectors and Matrixes.
- sreevisakh 5y agoAre you referring to eigen, the C++ library or eigen vectors/values, the mathematical construct?
- synergy20 5y agoFailed to see what chips are supported. Assuming x86 and ARM and their vector/SIMD are leveraged for performance, what about RISC-V's vector/SIMD, is there a plan to add those in the future?
- waynecochran 5y agoProbably won't ever catch Fortran, but using Eigen templates for reductions really opens the door for compile time optimizations; e.g. these are all reductions that do the same thing const float residual = (L.array() * P.array()).colwise().sum().square().mean(); const float residual = L.cwiseProduct(P).array().colwise().sum().array().square().mean(); const float residual = (L.transpose() * P).diagonal().array().square().mean(); The compiler can optimize using static information, e.g. these would all be handled differently for the following types Eigen::Matrix<float,3,3> L(3,3), P(3,3); Eigen::Matrix<float,3,Eigen::Dynamic> L(3,K), P(3,K); Eigen::Matrix<float,Eigen::Dynamic,Eigen::Dynamic> L(N,K), P(N,K);
- Symmetry 5y agoThe compile time loop fusion is also particularly nice.
- waynecochran 5y agoThanks -- I just learned something new. Since matrices are primitive types in Fortran, I assume these kinds of optimizations are more abundant in Fortran. I wonder if adding matrices / vectors / tensors as primitive types has even been entertained by the various C++ committees.
- markisus 5y agoEigen is really nice for getting code that looks like the underlying math and it also optimizes away unnecessary temporaries by using template expressions. However, when you mess up, you get a screen full of template errors that take some experience to understand and debug. It makes me wish for a language + compiler where linear algebraic objects are first-class values.
- tspooner 5y agoWhile it's not exactly what you're asking for, I do find the LinearAlgebra (standard) library in Julia to be pretty fantastic: https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/ https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/
- deng 5y ago> It makes me wish for a language + compiler where linear algebraic objects are first-class values. Like Fortran?
- adgjlsfhk1 5y agoFortran gives you first class dense ND-array, but doesn't give you the flexibility to make other types of arrays (banded/blocked/etc) feel first class.
- joelhaasnoot 5y agoAhh, the horror of getting this to compile last week in Python :(
- bobsomers 5y agoEigen is the standard choice (for good reason!) for many linear algebra projects, especially in robotics, but there is a big downside users should be aware of before they chose it. Eigen makes extensive use of expression templates in C++ to collapse complex operation sequences into streamlined and minimal calculations. This is generally ok, until you need a debug build. I've regularly seen debug builds of software using Eigen run 1000x to 10000x slower than the release build, which seriously complicates various debugging workflows. It also makes it a nightmare to run your test suite through valgrind, for example. I've seen several engineers attempt (and fail) to try creating/linking a release build of Eigen with a debug build of the rest of the program to try to regain most of that speed while still allowing a decent amount of debugability, but this is really hard due to all the aggressive inlining and heavy use of templates. In my experience, I would happily accept a 2x or more slowdown in linear algebra performance in release builds in exchange for significant boost in debug execution speed. If you're starting a greenfield project, you should consider how important decent debug performance is before choosing Eigen by default.
- hasmanean 5y agoThis is a problem with C++ in general. In debug mode it has no notion of performance whatsoever.
- gugagore 5y agoI appreciate being made aware of this downside! Why does this happen with C++ debugging?
- hermitdev 5y agoIn general, in template heavy libraries, what kills performance in debug builds is lack of inlining with optimizations turned off. And templates traditionally need to be inlinable by having their full definition in a header, so one can't even say build with optimizations on with debug info in isolation, because it has to be done where the templates are used, not in the library where they're defined (because there is no separately built library for the template). In other words, it's a real pain to get just the templates optimized while the rest of your code is not optimized for easier debugging.
- Communitivity 5y agoRecently encountered Eigen. It seems awesome - easy to work with, with lots of options. mat.conjugate() * mat.transpose() seems to take a while for my stuff, but I think I just haven't found the right method (next is to look at mat.adjointInPlace()).
- tgot 5y agoNot widely publicized, but the benchmarking code is in the source. At one point I was running it on my specific target machines to get performance estimates in support of porting some large-ish CPU stuff from Matlab into C++. The max performance was in Eigen-calling Intel MKL, but it was a big plus to not need MKL licenses on every development machine.
- lacker 5y agoIs there anything like Eigen for data that's kept on the GPU? I've been looking at porting some performance-critical scientific computing code from Python to C++ and numpy -> Eigen seems like an obvious migration path, but it's harder to figure out what to do with cupy matrix operations.
- ncmncm 5y agoThere is Kompute. It wraps Vulkan in a very thin layer, mostly to eliminate boilerplate.
- ogogmad 5y agoIt supports operator overloading too, by the way. This makes it usable where many LAPACK wrappers like Numpy are not.
- montecarl 5y agoOne of the best things about Eigen is that it is the only linear algebra package that I know of that easily supports using the same code for low, high, and arbitrary precision floating point numbers. I had some linear system of equations that I need to solve in grad school where the condition number was small enough that double precision was not sufficient to solve the problem. I was able to very easily use double double, quad double [1], and arbitrary precision floating [2] point number implementations to solve these problems. The matrices I used were not especially large, but I couldn't find any other existing packages that fit this use case. [1] https://www.davidhbailey.com/dhbsoftware/ https://www.davidhbailey.com/dhbsoftware/ [2] http://www.holoborodko.com/pavel/mpfr/ http://www.holoborodko.com/pavel/mpfr/
- gnufx 5y agoFor anyone still reading this: I'm now confused by Eigen reportedly being basically on a par with optimized L3 BLAS, which I assume doesn't just mean punting to optimized BLAS, which it seems it can do. I can't see any indication that you can do run-time dispatch on the micro-architecture, which I think is important; I saw very poor results until I examined CMakeLists.txt. Anyway, I cmade v3.4 with -DEIGEN_TEST_AVX512DQ=on, which seems most appropriate for my SKX system, and then did make blas. I ran my normal initial quick test using the OpenBLAS 0.3.15 dgemm benchmark, which reports a plateau of ~99000 MFlops. LD_PRELOADing libeigen_blas.so with the same binary under the same conditions (bound to a core, with the hpc-compute tuned profile) gave ~66000 MFlops. What did I do wrong?