4 ms·
From drewg123's other threads, it seems like their machines are also hitting (or approaching) memory bandwidth limits, so being able to reduce some of the memor
by toast0 5y ago
From drewg123's other threads, it seems like their machines are also hitting (or approaching) memory bandwidth limits, so being able to reduce some of the memory write/read requirements should help with that.
I think what was working best was having the disk I/O write to RAM that was NUMA aligned with the NIC, so disk -> PCIe -> NUMA Fabric -> RAM -> PCIe -> NIC.
If instead you could do disk -> PCIe -> NUMA Fabric -> PCIe -> NIC, at least for a portion of the disk reads, that would still be the same amount of traffic on the NUMA Fabric, but less traffic on the memory bus. This probably means that the NIC would be doing more high latency reads though, so you need more in-flight sends to keep throughput up.