You can run multi-node with tensor parallel plus pipeline parallel inference, e.g. with vLLM (https://docs.vllm.ai/en/latest/serving/distributed_serving.html ht

You can run multi-node with tensor parallel plus pipeline parallel inference, e.g. with vLLM (https://docs.vllm.ai/en/latest/serving/distributed_serving.html https://docs.vllm.ai/en/latest/serving/distributed_serving.h...).