3 ms·
That's what I was thinking. The slide seems to imply that workloads are becoming CPU bound due to IO speeds increasing a lot.
by azth 11y ago
That's what I was thinking. The slide seems to imply that workloads are becoming CPU bound due to IO speeds increasing a lot.
- zzalpha 11y agoThat's precisely what he's saying. This is the thesis of those who have been watching the explosion in solid state disks. The claim is that bulk I/O is becoming so damn fast that the pendulum is swinging toward computing power being the new limiting factor.
- vvanders 11y agoNo expert here on SSDs but it still seems like they're off by an order of magnitude. PCIe SSDs appear to top out around 1.8GB/sec while DDR4 is in the 12-19 GB/sec range. Either way CPU processing power has never really been a constraining factor(since most people don't want to write data aware code), it's always been bus-bound be it DRAM, IO or other.
- lmm 11y agoSpark is built around keeping things in-memory. If there's any workload that can max out a CPU (and surely there must be, otherwise no-one would bother building faster CPUs) then there's probably someone doing it on Spark (linear algebra? monte carlo simulation?). Spark might be a good fit with RDMA (looks like there's some initial work in that direction).