Intel adding instructions to do small matrix multiplies like Nvidia's tensor cores. With Winograd and Fourier convolution the natural unit of multiplication/acc

Intel adding instructions to do small matrix multiplies like Nvidia's tensor cores. With Winograd and Fourier convolution the natural unit of multiplication/accumulation is larger than a float (instead, 8 or 16 floats), so the computation does not "fit" the AMX model. (Here is a program that generates C intrinsic code to do tensor matrix multiplies with AVX-512: https://NN-512.com https://NN-512.com) "The Advanced Matrix Extension (AMX) is an x86 extension that introduces a new programming framework for working with matrices (rank-2 tensors). The extensions introduce two new components: a 2-dimensional register file with registers called 'tiles' and a set of accelerators that are able to operate on those tiles. The tiles represent a sub-array portion from a large 2-dimensional memory image. AMX instructions are synchronous in the instruction stream with memory load/store operations by tiles being coherent with the host's memory accesses. AMX instructions may be freely interleaved with traditional x86 code and execute in parallel with other extensions (e.g., AVX512) with special tile loads and stores and accelerator commands being sent over to the accelerator for execution." "The Tile Matrix Multiply (TMUL) Unit is an accelerator as part of AMX comprising a grid of fused multiply-add units capable of operating on tiles." "Its existence is defined by the AMX-INT8 and AMX-BF16 sub-extensions. The TMUL unit instruction set computes TileC[M][N] += TileA[M][K] * TileB[K][N]." "The TMUL unit comes with a number of parameters supported including the maximum height (tmul_maxk) and maximum SIMD dimension (tmul_maxn). Those parameters are dynamically read by the TMUL unit upon execution."