3 ms·
That's precisely what he's saying. This is the thesis of those who have been watching the explosion in solid state disks. The claim is that bulk I/O is becomi
by zzalpha 11y ago
That's precisely what he's saying.
This is the thesis of those who have been watching the explosion in solid state disks. The claim is that bulk I/O is becoming so damn fast that the pendulum is swinging toward computing power being the new limiting factor.
- vvanders 11y agoNo expert here on SSDs but it still seems like they're off by an order of magnitude. PCIe SSDs appear to top out around 1.8GB/sec while DDR4 is in the 12-19 GB/sec range. Either way CPU processing power has never really been a constraining factor(since most people don't want to write data aware code), it's always been bus-bound be it DRAM, IO or other.
- lmm 11y agoSpark is built around keeping things in-memory. If there's any workload that can max out a CPU (and surely there must be, otherwise no-one would bother building faster CPUs) then there's probably someone doing it on Spark (linear algebra? monte carlo simulation?). Spark might be a good fit with RDMA (looks like there's some initial work in that direction).