4 ms·
Whether you want a language to do this or not is a fundamental matter of design philosophy. I would be quite unhappy if a language changed multiplication by the
by StefanKarpinski 7y ago
Whether you want a language to do this or not is a fundamental matter of design philosophy. I would be quite unhappy if a language changed multiplication by the inverse of a matrix into backslash without my permission: that is not the computation I told you to do. In a language that did that automatically, how would you check the difference in the result between using the inverse and using backslash? You would try to compare them and conclude that they produce identical results, because the system had surreptitiously changed both computations to do the same thing behind your back. Then you’d get user reports about this and have to add a way to opt out of the optimization when it wasn’t wanted. And of course that’s not the only time you want to suppress an optimization like this—sometimes it will give incorrect results. Optimizations should either be truly indistinguishable from the correct answer or be opt-in.
In Julia we take a very different approach: make it as easy and convenient as possible to ask for the efficient version, but not automatic. For example, you can wrap a matrix that you know to be symmetric in the `Symmetric` wrapper type and leave the rest of the code the same and dispatch will call specialized BLAS and LAPACK routines for symmetric matrices. This approach had been designed by people who live and breathe numerical linear algebra and it allows them to write the naive version of an algorithm, and then tweak it to use the most efficient available kernels with minimal effort and without fundamentally changing the shape of their code.
What that means for the evaluation in this paper is that while the naive code may not do the fastest, cleverest thing, a very small variation on the naive code will do the fastest thing, using optimized kernels. You ask for this explicitly, but it’s easy and convenient to ask for. We’ve found this to be a very pragmatic and effective approach. It keeps optimization explicit but easy.
It would be an easy layer on top of that to add a code analyzer to look for patterns that might be possible to optimize further, but that seems like a problem that is better suited for tooling like a linter than being baked into the language itself.
- fyp 7y agoThe paper also talks about reordering matrix multiplications by automatically adding parentheses (despite the fact that floating point multiplication is non-associative). Is there a way to achieve this in Julia? If not what syntax might you introduce to do it? You can't really annotate just the matrices or multiplication operators since you are optimizing a whole expression.
- ChrisRackauckas 7y agoNeeds an update, but https://github.com/AustinPrivett/MatrixChainMultiply.jl https://github.com/AustinPrivett/MatrixChainMultiply.jl does it. Could probably be handled more automatically.
- photon-torpedo 7y agoMatrix chain multiplication can be seen a special case of tensor contraction, and the https://github.com/Jutho/TensorOperations.jl https://github.com/Jutho/TensorOperations.jl package has an implementation which tries to optimize the contraction order (macro @tensoropt).