5 ms·
How does it replicate all the data on disk or flash while maintaining the write latency of DRAM? I'm thinking there must be a delay during which loss of power
by kijin 8y ago
How does it replicate all the data on disk or flash while maintaining the write latency of DRAM?
I'm thinking there must be a delay during which loss of power will result in loss of data that was acknowledged as written. Is this something that RAMCloud overcomes in a novel way, or is the answer simply that the data is replicated across many nodes?
How about the problem of raw bandwidth? If you keep sending writes to a node that exceeds the bandwidth of the persistent storage medium but is well within the capabilities of DRAM, the storage medium will never be able to catch up even if we allow a generous delay. Maybe this is a non-issue right now because you can't send more than a few dozen Gbps over the network anyway, and we just need to hope that flash performance will improve faster than 100GbE goes mainstream.
- krageon 8y agoGiven the odds of these people inventing a huge solution to a hard problem in a publishing vacuum (I am not aware of papers coming out that fundamentally solve this, for example) I think it is pretty safe to assume it's subject to the same limitations all such systems are subject to. Loss of power leads to loss of data. Commonly mitigated as you already state.
- ernestoo 8y agoThe Raft Consensus Algorithm (https://raft.github.io/ https://raft.github.io/) was born out of this project
- glenngillen 8y agoSeems reasonable when the date on the OP is 9 years ago.
- mbreese 8y agoNot knowing anything about how the project works, they state their read latency is on the order of 5us and write latency is on the order of 15us. On this synopsis, they also mention that one mechanism they deal with is scale. So, I think your guess correct in that they likely replicate writes across multiple nodes. Then if any single node goes down, they still have a valid valid for a given key. Writes could then be persisted to disk asynchronously.
- Upvoter33 8y agoThey don't. Writes go to ONE server's memory. A crash will lose data. Speed over durability is the tradeoff. Eventually, writes are pushed to disk but not right away.