4 ms·
> Python/Numpy: 5360ms (Xeon(R) CPU E5-2698 v4 @ 2.20GHz) CuPy: 10.6ms (A100) MatX: 2.54ms (A100) > Are they even comparing apples to apples to cl
by elashri 3y ago
>
Python/Numpy: 5360ms (Xeon(R) CPU E5-2698 v4 @ 2.20GHz)
CuPy: 10.6ms (A100)
MatX: 2.54ms (A100)
>
Are they even comparing apples to apples to claim that they see these improvements over NumPy?
> While the code complexity and length are roughly the same, the MatX version shows a 2100x over the Numpy version, and over 4x faster than the CuPy version on the same GPU.
NumPy doesn't use GPU by default unless you use something like Jax [1] to compile NumPy code to run on GPUs. I think more honest comparison will mainly compare MatX running on same CPU like NumPy and for GPU comparison focus to compare vs CuPy.
[1] https://github.com/google/jax https://github.com/google/jax
- cburdick13 3y agoI agree we should update it given that it's also a very old comparison (2 years now). The cuPy comparison should be fair since it was the same GPU. I addressed it more here: https://news.ycombinator.com/item?id=37760120 https://news.ycombinator.com/item?id=37760120
- KeplerBoy 3y agoIt's very much an apples to oranges comparison, but nevertheless relevant, since NumPy is the defacto baseline for anything numeric in Python.
- albertzeyer 3y agoNot really, for GPU computation, PyTorch, TensorFlow or JAX are the baselines (they are not just for ML stuff). Even for CPU, that's a more relevant comparison.
- KeplerBoy 3y agoI agree in principle, but my experience is that researchers writing code just default to NumPy.
- zzbn00 3y agoThis seems like an FFT based benchmark.... I believe NumPy for licensing reasons includes a port of old FFTPACK which is only single threaded. (Py)FFTW or others are surely much faster. So the test seems to compare one thread of a CPU launched in 2016 running a port of 40year old fortran code to a optimised FFT library on a GPU launched in 2020.
- cburdick13 3y agoThe sample is really designed to show the simplicity of the syntax, and the performance is just a side effect. Where you'll see a bigger performance difference with numPy/cuPy is when kernel fusion happens where MatX is typically able to fuse many things into a single kernel at compile-time and cuPy launches many kernels. If you have a specific type of expression you'd like to compare please let us know.
- Night_Thastus 3y agoI would be curious how it compares to the big names in the field, Eigen, OpenBLAS, Boost, etc.
- cburdick13 3y agoHi, those libraries don't have a GPU counterpart, and for matrix multiplication we only support GPU right now.