3 ms·

How do you tune a CPU for inference?

• antoinealb 6y ago
  You usually tune the software that runs on it (compiler, libraries, build options etc.). You can also play the game of overclock if you run dedicated hardware.
  • DelightOne 6y ago
    I wondered what those options are. The other comment mention SIMD is interesting. Thanks :)
• deleted 6y ago
  [deleted]
• 37ef_ced3 6y ago
  You can generate custom SIMD inference code for your neural net (tensor shapes, etc.) and CPU cache hierarchy (L1, L2 per thread, etc.) For example, here is ResNet50 for a particular Skylake-X CPU: https://nn-512.com/browse/ResNet50 https://nn-512.com/browse/ResNet50
  • DelightOne 6y ago
    I didn't know this exists, thank you!