3 ms·
These CPUs are intended just to run miscellaneous tasks, such as loading AI models or running the cluster operating system. They don't need to be performant, ju
by edward28 2y ago
These CPUs are intended just to run miscellaneous tasks, such as loading AI models or running the cluster operating system. They don't need to be performant, just efficient, as the GPU does all the heavy lifting.
NVIDIA also provides an option to swap the grace chips out with an x86 chip, which could deliver better performance depending on the remaining power budget though.
- MobiusHorizons 2y agoIf this is all there is to it, why do they have the high frequency and high l3 cache? Those seem to be optimizing for something, not just a “good enough” configuration for a part that is not the bottleneck
- riotnrrd 2y agoData augmentation in CPU-space is often compute-light, but requires rapid access to memory. There are libraries (like NVIDIA's Dali) that can do augmentation on the GPU, but this takes up GPU resources that could be used by training. Having a multi-core CPU with fast caches is a good compromise.