No, the convolution generators were written to saturate the hardware For example, under ideal conditions (no out-of-register memory access, no waiting on depen

No, the convolution generators were written to saturate the hardware For example, under ideal conditions (no out-of-register memory access, no waiting on dependencies) you can sustain 27 single-precision FMADDs per CPU-core per cycle on a particular Skylake-X (i.e., approx 1.7 _mm512_fmadd_ps per cycle, each yielding 16 multiply-adds) As soon as you start accessing memory, that number drops to about 20 FMADDs. With direct convolution methods (1x1 and arbitrary), the best you can do is achieve that, and NN-512 comes close With the Fourier and Winograd convolutions, you start being limited by memory bandwidth, but the reduction in FMADDs that these methods provide means you end up ahead: your "effective" FMADD rate is much higher than what is possible through direct convolution. For example, NN-512 can exceed 48 effective FMADDs per cycle (on the 27 peak FMADD machine) with Winograd-Cook-Toom-Lavin, if the tensor is deep enough (enough channels) So, NN-512 succeeds in saturating the hardware. Essentially all the time is spent in the matrix multiplications, doing FMADDs, or being blocked bringing half-precision weights into register for Fourier or Winograd Until I generate a table of comparisons, you can use the previously stated number to do rough comparisons against the literature or other software packages: 18 DenseNet121 inferences per CPU-core per second on a cheap Skylake-X cloud instance