4 ms·
Ah, It was from an older implementation that wasn't compatible with Nim v2. I've commented it out. If you pull again it should work. > Anyway the reason for
by mratsim 3y ago
Ah,
It was from an older implementation that wasn't compatible with Nim v2. I've commented it out.
If you pull again it should work.
> Anyway the reason for your competitive performance is likely that you are benchmarking with very small matrices. OpenBLAS spends some time preprocessing the tiles which doesn't really pay off until they become really huge.
I don't get why you think it's impossible to reach BLAS speed. The matrix sizes are configured here: https://github.com/mratsim/laser/blob/master/benchmarks/gemm/gemm_bench_float32.nim#L53-L56 https://github.com/mratsim/laser/blob/master/benchmarks/gemm...
It defaults to 1920x1920 * 1920x1920. Note, if you activate the benchmarks versus PyTorch Glow, in the past it didn't support non-multiple of 16 or something, not sure today.
Packing is done here: https://github.com/mratsim/laser/blob/master/laser/primitives/matrix_multiplication/gemm_packing.nim https://github.com/mratsim/laser/blob/master/laser/primitive...
And it also support pre-packing which is useful to reimplement batch_matmul like what CuBLAS provides and is quite useful for convolution via matmul.
- bjourne 3y agoOk, I will benchmark this more when I have time. My gut feeling is that it is impossible to reach OpenBLAS-like performance without carefully tuned kernels and without explicit SIMD code. Clearly, it's not impossible to be as fast as OpenBLAS otherwise OpenBLAS itself woudln't be that fast, but it is very difficult and takes a lot of work. There is a reason much of OpenBLAS is implemented in assembly and not C.