If neural net inference is a small part of a larger computation, doing it locally (in cache) on the CPU (with AVX-512 instructions) can be a big win (for exampl

If neural net inference is a small part of a larger computation, doing it locally (in cache) on the CPU (with AVX-512 instructions) can be a big win (for example, https://NN-512.com https://NN-512.com)