For small convolutions on x86_64, you probably want libxsmm: https://libxsmm.readthedocs.io/en/latest/libxsmm_dl/ https://libxsmm.readthedocs.io/en/latest/libxs

For small convolutions on x86_64, you probably want libxsmm: https://libxsmm.readthedocs.io/en/latest/libxsmm_dl/ https://libxsmm.readthedocs.io/en/latest/libxsmm_dl/ It's not clear to me, from experience with generic kernels for GEMM, that you should second-guess the compiler (GCC, specifically) by hand-unrolling and attempting to hand-vetorize, unless you think its cost model is wrong. (Recent GCC does unroll-and-jam with -O3, and there is a pragma for unrolling.) Unrolling for vectorization may or may not be profitable in a particular case, depending on the SIMD length (e.g. -mavx2 v. -mavx512). You can see what GCC is up to with -fopt-info variants. The article says you can't explicitly load to cache, but prefetching may be be a significant benefit, as in OpenBLAS and BLIS kernels.