3 ms·

Simply storing the data in memory, with backup on disk, is not difficult or novel. The hard part is meeting the design's performance goals: "It should be possi

by neilc 17y ago

Simply storing the data in memory, with backup on disk, is not difficult or novel. The hard part is meeting the design's performance goals: "It should be possible to achieve access latencies of 5- 10 microseconds, measured end-to-end for a process running in an application server to read a few hundred bytes of data from a single record in a single storage server in the same datacenter." A single multi-core storage server should be able to service at least 1,000,000 small requests per second.

gruseom 17y ago
Hmm, but the paper mostly argues that that performance advantage comes automatically from using RAM instead of disk. The only obstacle they mention to realizing 5-10µs latency per se is that networking infrastructure can't yet support it. No question that's a big deal, but they spend far more time talking about durability, data models, distribution and scaling, all of which are auxiliary to raw latency. I'm not criticizing the paper, just seeking to understand how far the vision they outline really is from current practice. One interesting aspect is the sharp distinction the authors make between the approach they're advocating and systems that make heavy use of RAM caching (obviously common today), even when the RAM caches hold nearly all the data. So ok, let's rule out cache-based systems as instances of what they call RAMclouds. How widespread, today, are true RAM-based systems (as defined in the paper), even if they don't yet achieve 5-10µs round trips to storage? What major systems are in production that can be cited? Google were famous years ago for keeping their web indexes in RAM. Does that count? No question many innovative projects have sprouted up in the last few years in this space (e.g. Redis), and while they sound fabulous, I don't think they count as answers to my question without examples of major systems in production (for some fuzzy value of "major"). If anyone can cite any, please do. Troll disclaimer: Our startup is currently working on these very issues (storage strategy and how application talks to storage), so my interest is both genuine and acute. PS: Simply storing the data in memory, with backup on disk, is not difficult or novel. That's good to hear. What specifically are the common strategies for backup to disk?
- neilc 17y ago
  The only obstacle they mention to realizing 5-10µs latency per se is that networking infrastructure can't yet support it. The paper also talks about other issues that will need to be resolved (Section 4.1): software overheads like context switching and polling network devices, and protocol issues (e.g. TCP's minimum retransmission timeout is currently 200 milliseconds, which would make even a single dropped packet catastrophic for latency). I do agree that it is a bit suspect for the authors to go advocate at length for low-latency RPCs, but not volunteer to step up to the plate to do the innovation in switches and network hardware that this will require. What specifically are the common strategies for backup to disk? Well, most schemes follow some variant of write-ahead logging to a "stable" location: either local disk or to the RAM of a remote network node (and then assuming that the local and remote nodes won't fail simultaneously).

by neilc 17y ago

gruseom 17y ago

Hmm, but the paper mostly argues that that performance advantage comes automatically from using RAM instead of disk. The only obstacle they mention to realizing 5-10µs latency per se is that networking infrastructure can't yet support it. No question that's a big deal, but they spend far more time talking about durability, data models, distribution and scaling, all of which are auxiliary to raw latency. I'm not criticizing the paper, just seeking to understand how far the vision they outline really is from current practice. One interesting aspect is the sharp distinction the authors make between the approach they're advocating and systems that make heavy use of RAM caching (obviously common today), even when the RAM caches hold nearly all the data. So ok, let's rule out cache-based systems as instances of what they call RAMclouds. How widespread, today, are true RAM-based systems (as defined in the paper), even if they don't yet achieve 5-10µs round trips to storage? What major systems are in production that can be cited? Google were famous years ago for keeping their web indexes in RAM. Does that count? No question many innovative projects have sprouted up in the last few years in this space (e.g. Redis), and while they sound fabulous, I don't think they count as answers to my question without examples of major systems in production (for some fuzzy value of "major"). If anyone can cite any, please do. Troll disclaimer: Our startup is currently working on these very issues (storage strategy and how application talks to storage), so my interest is both genuine and acute. PS: Simply storing the data in memory, with backup on disk, is not difficult or novel. That's good to hear. What specifically are the common strategies for backup to disk?

neilc 17y ago
The only obstacle they mention to realizing 5-10µs latency per se is that networking infrastructure can't yet support it. The paper also talks about other issues that will need to be resolved (Section 4.1): software overheads like context switching and polling network devices, and protocol issues (e.g. TCP's minimum retransmission timeout is currently 200 milliseconds, which would make even a single dropped packet catastrophic for latency). I do agree that it is a bit suspect for the authors to go advocate at length for low-latency RPCs, but not volunteer to step up to the plate to do the innovation in switches and network hardware that this will require. What specifically are the common strategies for backup to disk? Well, most schemes follow some variant of write-ahead logging to a "stable" location: either local disk or to the RAM of a remote network node (and then assuming that the local and remote nodes won't fail simultaneously).