5 ms·
Once you factor the storage latency to/through the server the IOPS at realistic queue depths fall by an enormous factor. This is usually balanced by putting ton
by zamadatix 6y ago
Once you factor the storage latency to/through the server the IOPS at realistic queue depths fall by an enormous factor. This is usually balanced by putting tons of servers on it at which point 41 NVMe drives isn't really a lot - will usually fit in a single rack mount storage enclosure.
- Rafuino 6y agoThe issue though is that each server itself is starting to have 100 Gbit network connections, not just the rack itself.
- zamadatix 6y agoBandwidth isn't the problem, latency is. Each server could have 400 terabit interfaces and you'll still have garbage iops compared to local pcie on a laptop for reasonable depth.
- Rafuino 6y agoCan you help me understand why latency kills IOPS? There's lots of work showing NVMe over TCP bringing similar performance and latency from a remote server as compared to a local SSD. Here's an example: https://www.micron.com/about/blog/2020/march/nvme-over-tcp-proof-of-concept https://www.micron.com/about/blog/2020/march/nvme-over-tcp-p...