4 ms·
NVMe likely cannot scale to thousands of attached drives simultaneously due to limitations of PCI-E while SAS can. Scaling to thousands of attached disks was th
by ryao 11y ago
NVMe likely cannot scale to thousands of attached drives simultaneously due to limitations of PCI-E while SAS can. Scaling to thousands of attached disks was the motivation for QEMU's virtio-scsi because virtio-blk's 1 block device per PCI-E device meant that virtual machines using virtio-bulk could not scale past 32 drives. nVMe's 1 drive per PCI l-E device model should suffer from the same limitation.
Similarly, SAS supports multipath for HA while PCI-E does not appear to offer that ability. If it does, then I have yet to hear of an example of hardware implementing it.
Lastly, the idea is likely to appeal to those that care about density. For that purpose, it is better to lower costs and requiring a multitude of PCI express slots is not a great way of doing that. Motherboards with more PCI-E slots are expensive due to the layers needed to accommodate the circuits required by them. Assuming JBODs for such things exist (using a chip to share lanes among multiple devices), they would be expensive and also add to cost.
There is also a decent write up on this here:
http://www.datacenterjournal.com/nvme-connected-ssds/ http://www.datacenterjournal.com/nvme-connected-ssds/
- scurvy 11y agoNVMe doesn't require a PCIe slot. It just requires some lanes. You can use the 2.5" form factor with the same performance. It works really well. NVMe scales just fine. As for the livelihood of those selling these storage appliances, well check out NetApp stock price lately. Also,I wouldn't want to run thousands of these SAS drives in today's storage appliances. Even with high performance, the control plane and rebuild/rebalance stuff would be a nightmare. Trying to imagine a 50 node Isilon cluster with these things... not pretty.
- ryao 11y agoPeople are building ZFS systems with thousands of drives. As for being different than virtio because it needs PCI-E lanes, what keeps QEMU from emulating however many are needed?