2 ms·
Not shockingly, we've already moved beyond caring about local ram for many problems. Any system that is running distributed should be taking into account how l
by taeric 10y ago
Not shockingly, we've already moved beyond caring about local ram for many problems. Any system that is running distributed should be taking into account how long it takes to get data from across the network.
Similarly, the difficulty in large data problems is often more of getting the data in the first place. This is often about coordinating the data creation of many machines and systems.
So, for most of us, this shift will not be huge. Because it is mostly irrelevant. Even systems that do have this much data often benefit from algorithms that don't rely on directly accessing and processing all of it in one shot. If only for resilience. (That is, if a system failing doesn't mean reprocessing all of the data, restarting and recovery is often faster and easier to deal with.)
- digi_owl 10y ago"As above, so below" comes to mind. Sometimes it seems that the diff between a CPU and a cluster is the suffix put on the latency times.
- cowardlydragon 10y agoI thought I read somewhere that the fastest networking technology actually gets close to a couple hundred instructions window of processing. http://netoptimizer.blogspot.com/2014/05/the-calculations-10gbits-wirespeed.html http://netoptimizer.blogspot.com/2014/05/the-calculations-10... Hm, 67 ns per packet. So even this local storage would be the drag in many scenarios...