3 ms·
> Splitting the model up between several GPUs would add a third much worse bottleneck – memory bandwidth between the GPUs. What if you allowed the system to on
by amelius 2y ago
> Splitting the model up between several GPUs would add a third much worse bottleneck – memory bandwidth between the GPUs.
What if you allowed the system to only have a shared memory between every neighboring pair of GPUs?
Would that make sense for an LLM?