7 ms·

So what's special about this database? What properties does it have that are superior to the currently available alternatives? This article is all hype and no s

by eloff 12y ago

So what's special about this database? What properties does it have that are superior to the currently available alternatives? This article is all hype and no substance. EDIT: GitHub has more information. It's a scalable ordered key-value store (think a distributed version of Berkeley db.) Storage is based on RocksDB (a variant of LevelDB) and consensus is achieved using Raft. The database is written in Go. It's meant to "tolerate disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention." What's not clear at all is where it sits CAP wise. It says it's available and strongly consistent. Which would be CA, which is not an option (especially for something claiming to be failure tolerant.) It must either sacrifice write availability or consistency in the event of a partition. No idea which way it leans.

Goranek 12y ago
I think they're trying to create open source Spanner. Here is Google scientific paper about Spanner. http://static.googleusercontent.com/media/research.google.com/hr//archive/spanner-osdi2012.pdf http://static.googleusercontent.com/media/research.google.co...
- boomzilla 12y ago
  In my opinion, one of the most profound ideas in Spanner is the introduction of TrueTime API and the guarantee provided in the implementation. I wonder if this project is going to have something similar?
  - yid 12y ago
    From the description, it sounds like they're not at the moment. Instead, they seem to be aiming for the globally replicated, consistent, SQL-supporting features. Nothing wrong with that -- the world could use more geographically-aware database implementations. Seems like they'd be able to make use of the time sync for more efficient replication/failover/transactions in the future when the hardware is more widely available.
peterwwillis 12y ago
> Cockroach provides snapshot isolation (SI) and serializable snapshot isolation (SSI) semantics, allowing externally consistent, lock-free reads and writes--both from an historical snapshot timestamp and from the current wall clock time. SI provides lock-free reads and writes but still allows write skew. SSI eliminates write skew, but introduces a performance hit in the case of a contentious system. SSI is the default isolation; clients must consciously decide to trade correctness for performance. Cockroach implements a limited form of linearalizability, providing ordering for any observer or chain of observers. Either you use a strongly consistent mode that can have poor performance under contention, or a strongly consistent mode that will have good performance under contention but lots of failed transactions. So you get to decide on performance vs availability. To answer your question, it sacrifices availability, not consistency. It's an MVCC, after all.
jbooth 12y ago
Why are they using RocksDB rather than LMDB? If it's based on Raft, then it sacrifices availability if there aren't a quorum of nodes online.
- peterwwillis 12y ago
  Probably because 1. LMDB is limited to logical address space, 2. it has one big global lock, 3. It's a B-Tree, and both of those contribute to the fact that 4. LMDB is a read-oriented database [performance wise]. I would conjecture that Rocks could also be 'more easily embeddable', but i'm talking out my ass there :) And yeah, you kind of have to sacrifice availability if you want to stay consistent in the face of write skew...
  - jbooth 12y ago
    One big global write lock in LMDB maps pretty well to one single stream of replicated log entries in Raft, IMO. And logical address space is still far in excess of what most disks or arrays can fit, right? 40 bits or so on linux? EDIT: 47 bits, for 128TB -- http://stackoverflow.com/questions/2159456/whats-the-max-file-mapping-size-in-64bits-machine http://stackoverflow.com/questions/2159456/whats-the-max-fil...
    - peterwwillis 12y ago
      I don't see how? One global write lock means a single instance can't update multiple ranges at a time, so determining consensus and writing from multiple peers would just take a long time for no reason. The whole point of an SSI MVCC is to get around difficult locks.... If Moore's law holds, a single SSD will outgrow the address space in around 7 years. In four years, an array of eight disks would outgrow the address space. This is just for a single server. If you want a linearly-scaling, robust solution for future requirements (like multi-petabyte and exabyte distributed datastores), there's no reason to lock yourself into technology that'll be obsolete in half a decade. (edit: SanDisk says it may release 8TB SSDs next year, also adding "We see reaching the 4TB mark as really just the beginning and expect to continue doubling the capacity every year or two, far outpacing the growth for traditional HDDs")

by eloff 12y ago

Goranek 12y ago

I think they're trying to create open source Spanner. Here is Google scientific paper about Spanner. http://static.googleusercontent.com/media/research.google.com/hr//archive/spanner-osdi2012.pdf http://static.googleusercontent.com/media/research.google.co...

boomzilla 12y ago
In my opinion, one of the most profound ideas in Spanner is the introduction of TrueTime API and the guarantee provided in the implementation. I wonder if this project is going to have something similar?
- yid 12y ago
  From the description, it sounds like they're not at the moment. Instead, they seem to be aiming for the globally replicated, consistent, SQL-supporting features. Nothing wrong with that -- the world could use more geographically-aware database implementations. Seems like they'd be able to make use of the time sync for more efficient replication/failover/transactions in the future when the hardware is more widely available.

peterwwillis 12y ago

> Cockroach provides snapshot isolation (SI) and serializable snapshot isolation (SSI) semantics, allowing externally consistent, lock-free reads and writes--both from an historical snapshot timestamp and from the current wall clock time. SI provides lock-free reads and writes but still allows write skew. SSI eliminates write skew, but introduces a performance hit in the case of a contentious system. SSI is the default isolation; clients must consciously decide to trade correctness for performance. Cockroach implements a limited form of linearalizability, providing ordering for any observer or chain of observers. Either you use a strongly consistent mode that can have poor performance under contention, or a strongly consistent mode that will have good performance under contention but lots of failed transactions. So you get to decide on performance vs availability. To answer your question, it sacrifices availability, not consistency. It's an MVCC, after all.

jbooth 12y ago

Why are they using RocksDB rather than LMDB? If it's based on Raft, then it sacrifices availability if there aren't a quorum of nodes online.

peterwwillis 12y ago
Probably because 1. LMDB is limited to logical address space, 2. it has one big global lock, 3. It's a B-Tree, and both of those contribute to the fact that 4. LMDB is a read-oriented database [performance wise]. I would conjecture that Rocks could also be 'more easily embeddable', but i'm talking out my ass there :) And yeah, you kind of have to sacrifice availability if you want to stay consistent in the face of write skew...
- jbooth 12y ago
  One big global write lock in LMDB maps pretty well to one single stream of replicated log entries in Raft, IMO. And logical address space is still far in excess of what most disks or arrays can fit, right? 40 bits or so on linux? EDIT: 47 bits, for 128TB -- http://stackoverflow.com/questions/2159456/whats-the-max-file-mapping-size-in-64bits-machine http://stackoverflow.com/questions/2159456/whats-the-max-fil...
  - peterwwillis 12y ago
    I don't see how? One global write lock means a single instance can't update multiple ranges at a time, so determining consensus and writing from multiple peers would just take a long time for no reason. The whole point of an SSI MVCC is to get around difficult locks.... If Moore's law holds, a single SSD will outgrow the address space in around 7 years. In four years, an array of eight disks would outgrow the address space. This is just for a single server. If you want a linearly-scaling, robust solution for future requirements (like multi-petabyte and exabyte distributed datastores), there's no reason to lock yourself into technology that'll be obsolete in half a decade. (edit: SanDisk says it may release 8TB SSDs next year, also adding "We see reaching the 4TB mark as really just the beginning and expect to continue doubling the capacity every year or two, far outpacing the growth for traditional HDDs")