4 ms·
Is there a linux equivalent of this setup? I see some mention of RDNA support for linux distros, but it's unclear to me if this is hardware-specific (requires
by smlacy 10mo ago
Is there a linux equivalent of this setup? I see some mention of RDNA support for linux distros, but it's unclear to me if this is hardware-specific (requires ConnectX or in this case Apple Thunderbolt) or is there something interesting that can be done with "vanilla 10G NIC" hardware?
- deleted 10mo ago[deleted]
- Maxious 10mo agoTo get the production level performance, you do need the RDNA compatible hardware. However, vLLM supports multi node clusters over normal ethernet too https://docs.vllm.ai/en/stable/serving/parallelism_scaling/#multi-node-deployment https://docs.vllm.ai/en/stable/serving/parallelism_scaling/#...