3 ms·

Eventual Consistency and Transactions Working Together

cblock811 12y ago
Are there any other databases that can do what you're describing here?
- topher-the-geek 12y ago
  Not that I'm aware of. CockroachDB and FoundationDB provide scalable and reliable transactions. But they don't offer this interface. Couchbase has conditional writes, but not batch writes. I think TreodeDB is the first to bring these things together this way.
- emin-gun-sirer 12y ago
  HyperDex provides scalable and high-performance transactions. While it is strongly (not eventually) consistent, the underlying technique we use to achieve atomicity of transactions is reminiscent of this approach. On a related note, our experience is that a tightly implemented database can be much faster than a typical eventually consistent database. It would be great to see performance comparisons from TreodeDB.
  - topher-the-geek 12y ago
    That's cool. Indeed, it does look like one could use HypderDex's transactions and cond_put to make something close the batch write described in this post. However, I don't see the part about Lamport clocks in HyperDex's API. Maybe one could implement that in a layer on top. The technique in the post allows a client to know if the values in its cache satisfy application invariants, even invariants that relate multiple rows. The Lamport clocks are key to making that part work. I would very much like to provide performance numbers, and Jepsen results, and more for TreodeDB. Along that line though, I'm looking for contributors (or even a co-founder). I've gotten it this far on my own, but I really need help.
jakequist 12y ago
Reminds me of Spanner with some changes. http://static.googleusercontent.com/media/research.google.com/en/us/archive/spanner-osdi2012.pdf http://static.googleusercontent.com/media/research.google.co...
- topher-the-geek 12y ago
  TreodeDB a bit like Spanner, in as much as both offer transactions across datacenters. The implementations are very different. The way Treode uses Lamport clocks lets you use a simple rule to determine if your local cache satisfies application invariants. I'm not sure if Spanner's use of atomic clocks gets you a similar capability.

Are there any other databases that can do what you're describing here?

topher-the-geek 12y ago
Not that I'm aware of. CockroachDB and FoundationDB provide scalable and reliable transactions. But they don't offer this interface. Couchbase has conditional writes, but not batch writes. I think TreodeDB is the first to bring these things together this way.
emin-gun-sirer 12y ago
HyperDex provides scalable and high-performance transactions. While it is strongly (not eventually) consistent, the underlying technique we use to achieve atomicity of transactions is reminiscent of this approach. On a related note, our experience is that a tightly implemented database can be much faster than a typical eventually consistent database. It would be great to see performance comparisons from TreodeDB.
- topher-the-geek 12y ago
  That's cool. Indeed, it does look like one could use HypderDex's transactions and cond_put to make something close the batch write described in this post. However, I don't see the part about Lamport clocks in HyperDex's API. Maybe one could implement that in a layer on top. The technique in the post allows a client to know if the values in its cache satisfy application invariants, even invariants that relate multiple rows. The Lamport clocks are key to making that part work. I would very much like to provide performance numbers, and Jepsen results, and more for TreodeDB. Along that line though, I'm looking for contributors (or even a co-founder). I've gotten it this far on my own, but I really need help.

Reminds me of Spanner with some changes. http://static.googleusercontent.com/media/research.google.com/en/us/archive/spanner-osdi2012.pdf http://static.googleusercontent.com/media/research.google.co...

topher-the-geek 12y ago
TreodeDB a bit like Spanner, in as much as both offer transactions across datacenters. The implementations are very different. The way Treode uses Lamport clocks lets you use a simple rule to determine if your local cache satisfies application invariants. I'm not sure if Spanner's use of atomic clocks gets you a similar capability.