4 ms·
You are both correct. AI inference is a comparatively easy problem from the perspective of parallelization when compared to most HPC problems.
by pclmulqdq 2y ago
You are both correct. AI inference is a comparatively easy problem from the perspective of parallelization when compared to most HPC problems.
- ryao 2y agoFacebook did a technical paper where they described their training cluster and the sheer amount of complexity is staggering. That said, the original poster was interested in inferencing, not training. https://arxiv.org/pdf/2407.21783 https://arxiv.org/pdf/2407.21783
- pclmulqdq 2y agoTraining is close to traditional HPC in many ways. Inference is far simpler since it's a simple forward-going pipeline of a relatively small working set.