3 ms·

I haven't seen this website before and, after taking a cursory look at this and the previous posts mentioned the article, I'm highly skeptical of the quality of

I haven't seen this website before and, after taking a cursory look at this and the previous posts mentioned the article, I'm highly skeptical of the quality of this work: - In the C article he shows that LAPACK can be massively improved upon. That sounds very impressive, until you realize he hasn't mentioned which LAPACK/BLAS implementation he used. The default download from netlib is the vanilla version and is known to be very slow in general. Any benchmark dissing LAPACK is incomplete without comparison with OpenBLAS and Intel MKL. And instead we see the author pulling all sorts of overengineered tricks to top the performance of a vanilla LAPACK build (I mean there isn't even a mention of whether or not he used optimization flags while compiling LAPACK). - In this python aritcle, again, python comes with highly developed scipy/numpy stack which can be coupled with a highly performant OpenBLAS or Intel MKL, but there is absolutely zero mention of the most popular scientific stack in the python community. Again, instead, we see a round-about and "clever" solution of trying to couple python with LLVM.

math_and_stuff 8y ago
It's even worse than that. He's implemented Cramer's rule.
twtw 8y ago
On my system, I see the following: $ /usr/bin/time -f "%U\n" ./exp_volatile_a_b 0.13 $ /usr/bin/time -f "%U\n" ./exp_lapack 0.52 $ /usr/bin/time -f "%U\n" ./exp_openblas 0.32 I got LAPACK and BLAS via apt-get on Ubuntu 18.04, so whatever that means. I installed OpenBLAS from source, and it looks like the target detection decided to enable AVX2 (but not AVX512). So the fully-unrolled version is still faster on the 5x5 matrix, which doesn't seem super surprising to me. I would expect a LAPACK implementation to have some overhead compared to a straight line solution to a problem this small.
- celrod 8y ago
  Yes. This is easy to see in Julia, by comparing native arrays with StaticArrays.jl. Native arrays operations are linked to OpenBLAS, while many StaticArray operations are unrolled (will have to get to a computer to see if that includes 5x5 inversion). At the small sizes, BLAS is tens of times slower. Especially OpenBLAS (MKL does better at small sizes). There's a lot of overhead from things like determining optimal blocking structures.

math_and_stuff 8y ago

It's even worse than that. He's implemented Cramer's rule.

twtw 8y ago

On my system, I see the following: $ /usr/bin/time -f "%U\n" ./exp_volatile_a_b 0.13 $ /usr/bin/time -f "%U\n" ./exp_lapack 0.52 $ /usr/bin/time -f "%U\n" ./exp_openblas 0.32 I got LAPACK and BLAS via apt-get on Ubuntu 18.04, so whatever that means. I installed OpenBLAS from source, and it looks like the target detection decided to enable AVX2 (but not AVX512). So the fully-unrolled version is still faster on the 5x5 matrix, which doesn't seem super surprising to me. I would expect a LAPACK implementation to have some overhead compared to a straight line solution to a problem this small.

celrod 8y ago
Yes. This is easy to see in Julia, by comparing native arrays with StaticArrays.jl. Native arrays operations are linked to OpenBLAS, while many StaticArray operations are unrolled (will have to get to a computer to see if that includes 5x5 inversion). At the small sizes, BLAS is tens of times slower. Especially OpenBLAS (MKL does better at small sizes). There's a lot of overhead from things like determining optimal blocking structures.