4 ms·

Since the early 90's but in C++ through TMP and class template specializations. If you don't have many specializations, and those are sharing most of the code,

Since the early 90's but in C++ through TMP and class template specializations. If you don't have many specializations, and those are sharing most of the code, you can simplify writing this code through if-constexpr since C++17. But soon after you will realize that you need to do some operations with those tensors such as A + B + C. To compute this expression compiler will first need to compute (A + B) sub-expression and store the result into the temporary tensor. Only then it will be able to use this temporary result to compute the final expression by adding the C tensor. When you have a lot of such expressions to compute, and expressions are arbitrarily complex, you will want to avoid inefficiencies caused by creating many temporaries - there's an attached cost to them both in memory and in compute. And sometimes you will be able to fuse multiple operations into a single one. This is a problem that can be solved with lazy evaluation and in C++ this can be done in compile-time through expression templates. And this is what basically all linear algebra C++ libraries are famous of doing.

panzi 3y ago
In Rust you can implement Add for all kinds of combinations of types. I.e. when one of the parameters moves it's ownership you can just perform the operation in-place in that parameter and return it. So you can eliminate temporaries at compile time. It means much more work in the library, but the library user can just write A + B + C. So Rust has that and C++ has template specialization, const if, and variadic templates. Would be cool to have all in one language.
- menaerus 3y ago
  But we already do. > So you can eliminate temporaries at compile time. > ... but the library user can just write A + B + C ... > So Rust has that. Eliminating temporaries in C++ is also done in compile-time through expression templates or more generally through TMP. I'm pretty sure Rust type system is incapable to express many (?) of the TMP things from C++ - it does not even support partial specializations to begin with so it's hard to imagine that you can do a lot of TMP gymnastics. I think that this is an area where it is generally accepted that Rust is lacking against C++.
  - panzi 3y ago
    Yeah, I was listing a lot of things that C++ can express that Rust can't. But C++ doesn't have lifetimes so you can't express those. And you can use that to eliminate temporaries. But if you say that you can also somehow do that with template meta programming then great!

panzi 3y ago

In Rust you can implement Add for all kinds of combinations of types. I.e. when one of the parameters moves it's ownership you can just perform the operation in-place in that parameter and return it. So you can eliminate temporaries at compile time. It means much more work in the library, but the library user can just write A + B + C. So Rust has that and C++ has template specialization, const if, and variadic templates. Would be cool to have all in one language.

menaerus 3y ago
But we already do. > So you can eliminate temporaries at compile time. > ... but the library user can just write A + B + C ... > So Rust has that. Eliminating temporaries in C++ is also done in compile-time through expression templates or more generally through TMP. I'm pretty sure Rust type system is incapable to express many (?) of the TMP things from C++ - it does not even support partial specializations to begin with so it's hard to imagine that you can do a lot of TMP gymnastics. I think that this is an area where it is generally accepted that Rust is lacking against C++.
- panzi 3y ago
  Yeah, I was listing a lot of things that C++ can express that Rust can't. But C++ doesn't have lifetimes so you can't express those. And you can use that to eliminate temporaries. But if you say that you can also somehow do that with template meta programming then great!