3 ms·
The problem is not the CPU per se, but the PCI bandwidth congestion between CPU<->PCI lanes. These NVME drives can talk directly to each other for raid which m
by everfrustrated 2y ago
The problem is not the CPU per se, but the PCI bandwidth congestion between CPU<->PCI lanes.
These NVME drives can talk directly to each other for raid which means a much larger total bandwidth is available, and potentially improved latency also.
RDMA means you might not be serving via CPU at all.
- wmf 2y agoPCIe P2P transfers go through the CPU so... no? I think what it's saving is main memory bandwidth.
- deleted 2y ago[deleted]
- zamadatix 2y ago> PCIe P2P transfers go through the CPU PCIe P2P transfers can go direct through a downstream PCIe switch such as found on chipsets without having to bump back up through the CPU.
- namibj 2y agoAnd even with e.g. AMD Matisse, aka Desktop Ryzen Zen3 on AM4, it turns around in the PCIe root complex instead of consuming infinity fabric bandwidth.
- sliken 2y agoThe difference isn't that big though. Sure with software RAID you write 1GB to a 8 disk RAID6 you write 8/6 x 1GB = 1.33GB. But with RAID offload you nearly double the NVME bandwidth (n - 1) x 2 consumed. I also wonder, if you have 8 NVMe, write to a stripe to one, it does the RAID calc and sends each disk the share of the stripe. What happens if the master NVMe dies? It's not really a RAID if a single disk can kill the RAID.