You can generate custom SIMD inference code for your neural net (tensor shapes, etc.) and CPU cache hierarchy (L1, L2 per thread, etc.) For example, here is Re

You can generate custom SIMD inference code for your neural net (tensor shapes, etc.) and CPU cache hierarchy (L1, L2 per thread, etc.) For example, here is ResNet50 for a particular Skylake-X CPU: https://nn-512.com/browse/ResNet50 https://nn-512.com/browse/ResNet50