5 ms·

I might be misreading it, but it's a 2x25GbE connection to each drive. So 24x 25GbE in that one switch chassis. The network-switch-chassis has 6x 100GbE (or i

I might be misreading it, but it's a 2x25GbE connection to each drive. So 24x 25GbE in that one switch chassis. The network-switch-chassis has 6x 100GbE (or is it actually 200GbE) for external connections. Since the drives are dual-port, the other chassis has another 6x100/200GbE if you wanted to split traffic over the second disk port + network-switch-chassis. Upsides seem to be: - Can scale up storage by plugging in another switch/disk shelf - No storage-enforced limits on the number or type of arrays - Can define namespaces on drives so you can adjust storage to clients based on need. - Can scale up performance for a single client by making more drives available and/or faster client NICs - Not limited to the number of PCI lanes in the client so you could have fast access to FAR more disks than you could physically plug into one host. Performance would be limited to the client's NIC(s). At the end of the video Patrick talks about that while they demo it being done directly on a host, they can also have DPU cards that could do all of the mounting/array management bits. There's a video on that coming soon.

rasz 5y ago
"Marvell 98EX5630 Ethernet switch, 6x 100GbE on the rear of the chassis = 600Gbps. 25GbE for each drive * 24 drives = 600Gbps providing a 1:1 ratio" Whats the appeal of proprietary drives talking to fiber switch directly versus NVME chassis with 3 2x100G PCIe NICs on the back? From logical point of view there is no difference. You can parturition, provision, bond it up just the same. You just move PCIE-fiber conversion from central point (cheaper) to bespoke solution inside proprietary drives (sounds expensive vendor lock-in).
- paranoidrobot 5y ago
  I don't think these are proprietary drives or connectors. I can't see anything there to indicate it is proprietary. I think Marvell just happens to be the first out with the NVMEoF drive adapter and switch chips. Kioxia and the switch vendor seem to be first out with the drives and switches using Marvell chips/boards. I wouldn't be surprised to hear in a year or two that you can buy a NVMEoF switch from a bunch of vendors, and then NVMEoF drives from your choice of drive vendor. The specifics of advantages over existing solutions I can't speak to except repeating much of what is in the article and video. I might be placing too much faith in Patrick + STH, but they seem excited, and other times there have been proprietary solutions for things they have pointed it out.
- rektide 5y ago
  A nas like you propose is a bunch of drives, connected to a cpu, connected to a nic. Which then talks to a switch that the actual box using the drive connects to. This solution proposes dropping a lot of hardware & getting the same outcome. There's one box with 12x 100Gbe on the back (across two switches) and a bunch of nvme (over rdma ethernet, "NVMeoF") plugged directly into that switch. Why bother having a whole cpu & expensive nic just to connect some drives to the network? This box replaces not just that nas, but perhaps the top of switch box too. This isnt supposed to be bespoke or proprietary. Its supposed to just the infra we already have. If your nas is really good it already spoke NVMeoF anyhow. Whats a little fancy is that you still need some PHY between switch and drive, which i think is throwing you off & confusing you a bit & making you think this is more bespoke than it is. The drive is an ethernet drive, could hardly be less bespoke.

rasz 5y ago

"Marvell 98EX5630 Ethernet switch, 6x 100GbE on the rear of the chassis = 600Gbps. 25GbE for each drive * 24 drives = 600Gbps providing a 1:1 ratio" Whats the appeal of proprietary drives talking to fiber switch directly versus NVME chassis with 3 2x100G PCIe NICs on the back? From logical point of view there is no difference. You can parturition, provision, bond it up just the same. You just move PCIE-fiber conversion from central point (cheaper) to bespoke solution inside proprietary drives (sounds expensive vendor lock-in).

paranoidrobot 5y ago
I don't think these are proprietary drives or connectors. I can't see anything there to indicate it is proprietary. I think Marvell just happens to be the first out with the NVMEoF drive adapter and switch chips. Kioxia and the switch vendor seem to be first out with the drives and switches using Marvell chips/boards. I wouldn't be surprised to hear in a year or two that you can buy a NVMEoF switch from a bunch of vendors, and then NVMEoF drives from your choice of drive vendor. The specifics of advantages over existing solutions I can't speak to except repeating much of what is in the article and video. I might be placing too much faith in Patrick + STH, but they seem excited, and other times there have been proprietary solutions for things they have pointed it out.
rektide 5y ago
A nas like you propose is a bunch of drives, connected to a cpu, connected to a nic. Which then talks to a switch that the actual box using the drive connects to. This solution proposes dropping a lot of hardware & getting the same outcome. There's one box with 12x 100Gbe on the back (across two switches) and a bunch of nvme (over rdma ethernet, "NVMeoF") plugged directly into that switch. Why bother having a whole cpu & expensive nic just to connect some drives to the network? This box replaces not just that nas, but perhaps the top of switch box too. This isnt supposed to be bespoke or proprietary. Its supposed to just the infra we already have. If your nas is really good it already spoke NVMeoF anyhow. Whats a little fancy is that you still need some PHY between switch and drive, which i think is throwing you off & confusing you a bit & making you think this is more bespoke than it is. The drive is an ethernet drive, could hardly be less bespoke.