3 ms·
> You could very easily reach 100 GByte/s read which no network will match. High end networking gear already has higher throughput: http://www.mellanox.com/pa
by justinclift 7y ago
> You could very easily reach 100 GByte/s read which no network will match.
High end networking gear already has higher throughput:
http://www.mellanox.com/page/ethernet_cards_overview http://www.mellanox.com/page/ethernet_cards_overview
https://www.servethehome.com/mellanox-connectx-6-brings-200gbe-and-hdr-infiniband-fabric-to-hpc/ https://www.servethehome.com/mellanox-connectx-6-brings-200g...
A SAN/NAS using the same PCIe 4.0 SSD's you mention could probably fill the pipes too.
... and it would probably need a bunch of network stack tuning. ;)
- tigershark 7y agoThe 200 gb/s network adapter that you linked are 4 times slower than 100GB/s. The parent comment wrote explicitly 100 Gbyte/s.
- justinclift 7y agoOops. Didn't spot that, sorry. :) That being said, after re-reading the comment they're talking about adding multiple PCIe cards to a box to achieve 100GB/s of local total throughput. That would be achievable over a network by adding multiple PCIe 200Gb/s network cards too. :)
- chx 7y agoNah, a motherboard with enough M.2 connectors could easily exist. Or, U.2 or OCuLink. We have already seen 1P EPYC servers with six OCULink connectors...
- justinclift 7y agoSure. My point is just that whatever bandwidth you can do locally, you can also do over the network. As a sibling comment mentions though... the cost difference would be substantial. :(
- ti_ranger 7y ago>> You could very easily reach 100 GByte/s read which no network will match. > High end networking gear already has higher throughput: 100GB/s > 200Gbps You would need 4x 200Gbps ports to reach 100GB/s, so 2x MCX653105A-ECAT (each 2x 16-lanes) at > $700 each, and pay for 1/10th of a ~$30 000 switch, IOW 100GB/s would cost you ~ $4400, before paying for the storage. Sure, it could be done, but it wouldn't be cheap, and you'll have used most of the PCIe lanes.
- justinclift 7y agoAgreed. Higher end network gear is $$$. :( EPYC servers (128 PCIe lanes) would probably be the go too, not Xeons. This is just Imagineering though. ;) With the specifics, wouldn't it be 4 cards needed? Each card has 2x 100Gb/s ports, so 8 ports in total.