2 ms·
NVLink is most needed for training. For inference a lot of the popular models can usefully be run on multiple GPUs without it: https://www.reddit.com/r/LocalLL
by AnthonyMouse 3y ago
NVLink is most needed for training. For inference a lot of the popular models can usefully be run on multiple GPUs without it:
https://www.reddit.com/r/LocalLLaMA/comments/142rm0m/llamacpp_multi_gpu_support_has_been_merged/ https://www.reddit.com/r/LocalLLaMA/comments/142rm0m/llamacp...