You just copy data over PCIe, however it is slower than nvlink. There are several ways to utilize multiple GPUs, the main contenders as I understand it is pipel

You just copy data over PCIe, however it is slower than nvlink. There are several ways to utilize multiple GPUs, the main contenders as I understand it is pipelined and so-called tensor parallelism. Think of it as slicing a loaf of bread in regular slices vs along its longer axis. The former can have higher peak throughput while the latter can have lower latency, though it depends on the details[1]. [1]: https://blog.squeezebits.com/vllm-vs-tensorrtllm-9-parallelism-strategies-36310 https://blog.squeezebits.com/vllm-vs-tensorrtllm-9-paralleli...