We inherit any multi-host support from vLLM, so https://docs.vllm.ai/en/latest/serving/distributed_serving.html#running-vllm-on-multiple-nodes https://docs.vllm

We inherit any multi-host support from vLLM, so https://docs.vllm.ai/en/latest/serving/distributed_serving.html#running-vllm-on-multiple-nodes https://docs.vllm.ai/en/latest/serving/distributed_serving.h... would be the expected path. We plan to publish examples of multi-host inference that leverages LeaderWorkerSets - https://github.com/kubernetes-sigs/lws https://github.com/kubernetes-sigs/lws - which helps run ranked serving workloads across hosts. LeaderWorkerSet is how Google supports both TPU and GPU multi-host deployments - see https://github.com/kubernetes-sigs/lws/blob/main/config/samples/leaderworkerset_tpu.yaml https://github.com/kubernetes-sigs/lws/blob/main/config/samp... for an example. Edit: Here is an example Kubernetes configuration running DeepSeek-R1 on vLLM multi-host using LeaderWorkerSet https://github.com/kubernetes-sigs/wg-serving/blob/main/serving-catalog/core/lws/vllm/deepseek-r1/gke/README.md https://github.com/kubernetes-sigs/wg-serving/blob/main/serv.... This work would be integrated into llm-d.