4 ms·
Network bandwidth is not 20x storage ant more. An SSD is around 10GB/s now, so similar to 100Gb ethernet.
by justincormack 11mo ago
Network bandwidth is not 20x storage ant more. An SSD is around 10GB/s now, so similar to 100Gb ethernet.
- wtallis 11mo agoAnd that's for one SSD. If you're running on a server rather than a laptop, aggregate storage bandwidth will almost certainly be higher than any single network link.
- mrlongroots 11mo agoThe appropriate comparison point for aggregate cluster storage bandwidth would be its bisection bandwidth. (I do HPC, IIRC ANL Aurora is < 1PB/s DAOS and 20 PB/s bisection).
- mrlongroots 11mo agoI think I'm talking about cluster-scale network bisection bandwidth vs attached storage bandwidth. With replication/erasure coding overhead and the economics, the order of magnitude difference still prevails. I think your point is a good one in that it is more economics than systems physics. We size clusters to have more compute/network than storage because it is the design point that maximizes overall utility. I think it also raises an interesting question in that let's say we get to a point where the disparity really no longer holds: that would justify a complete rethinking of many Spark-like applications that are designed to exploit this asymmetry.