3 ms·
> you need a buffer somewhere, and there is no place for the buffer. > The alternative is to have a box of RAM sitting on the PCIe bus. One thing I didn't thi
by toast0 5y ago
> you need a buffer somewhere, and there is no place for the buffer.
> The alternative is to have a box of RAM sitting on the PCIe bus.
One thing I didn't think of until now is your 400 Gbps system uses effectively 32 PCI-e 4.0 lanes of Nic and 32 more lanes of storage (i think it's 64 lanes of 3.0 storage in system), and you mention elsewhere that your 800 Gbps system needs 64 lanes of NIC and about 64 lanes of storage, and that you're having a hard time finding that many lanes on a board that meets your requirements. You'd need more lanes for the PCI-e ram disk, even if you could find one more affordable and available than GPUs.
Sounds like Mellanox (or someone) needs to add some ram to their NICs. (Or wait for PCI-e 5.0 and DDR5)