OK that's pretty fascinating, turns out Mooncake includes a trick that can populate GPU VRAM directly from NVMe SSD without it having to go through the host's r

OK that's pretty fascinating, turns out Mooncake includes a trick that can populate GPU VRAM directly from NVMe SSD without it having to go through the host's regular CPU and RAM first! https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/transfer-engine.md#2-nvmeof-segment https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/tran... > Transfer Engine also leverages the NVMeof protocol to support direct data transfer from files on NVMe to DRAM/VRAM via PCIe, without going through the CPU and achieving zero-copy.