6 ms·

It's interesting to see that the creators of BigTable and the early proponents of eventual consistency have invested the last 4.5 years building a system that a

It's interesting to see that the creators of BigTable and the early proponents of eventual consistency have invested the last 4.5 years building a system that adds back strong consistency guarantees. If the Spanner paper is as important as BigTable, ACID may become the new goal for those building distributed systems. Full disclosure: I'm with FoundationDB, which is a distributed NoSQL database with high performance cross-node ACID transactions. http://www.foundationdb.com http://www.foundationdb.com

shin_lao 14y ago
From what I understand of this specific project, they "solved" the consistency problem (which is quite a feat, granted). They don't say they have an ACID NoSQL database, which is, to me, an oxymoron: ACIDity is useful if you have a powerful query language. In the end don't you fear that you might simply reinvent SQL? Or, am I missing something?
- Evbn 14y ago
  Which part of acid is relevant specifically to SQL (joins, I assume you mean)?
- nlavezzo 14y ago
  ACID and NoSQL are not mutually exclusive, although that is a common misconception due to the fact that almost all NoSQL databases gave up ACID in exchange for BASE (eventual consistency), making the development of their distributed databases much easier. However, ACID is simply a set of properties that allows you to group a set of reads & writes to a database into a transaction that is Atomic, Consistent, Isolated, and Durable. That set of reads & writes does not need to be SQL - in our case, it is a set of reads from and writes to a database of ordered keys and values. For example, you could read the values associated with keys "a" and "b" and "c" and based on that information, make a change to the values associated with keys "c" and "d" and then commit the transaction. The transaction then either fails completely (if one of the keys you read had been written to since your read, in which case you re-try the transaction) or succeeds completely. That is an ACID transaction in a (NoSQL) key-value store.
  - shin_lao 14y ago
    That sounds useful, but not as useful as transactions on joins. It's great to have transactions when you have to cross reference data, on a key/value store, although you can have range queries I submit that kind of operations is less frequent. But still, it sure is great to be able to run a batch of operations with the confidence it will be transactional, I'm sure there are many use cases that can benefit from it. Oddly enough, it's strange that there aren't more NoSQL engines offering this feature as once you have MVCC you've done the hard part and AFAIK several NoSQL db have MVCC.
    - nlavezzo 14y ago
      Yep, those types of operations are less frequent in k/v stores, but we believe it is because of the lack of ACID in most of them. When you have ACID and ordering you can use transactions to keep data and multiple indexes on that data in synch, and use range operations to query the indexes. You can build all manner of higher level data structures with a transactional ordered k/v store, which is why our concept of "layers" is possible: www.foundationdb.com/#layers
- damian2000 14y ago
  It seems to me more like they have "solved" (or broken) the CAP theorem ... http://en.wikipedia.org/wiki/CAP_theorem http://en.wikipedia.org/wiki/CAP_theorem * Consistency (all nodes see the same data at the same time) * Availability (a guarantee that every request receives a response about whether it was successful or failed) * Partition tolerance (the system continues to operate despite arbitrary message loss or failure of part of the system) According to the theorem, a distributed system can satisfy any two of these guarantees at the same time, but not all three.
  - snewman 14y ago
    No, they haven't broken the CAP theorem. If the network were to partition badly enough, Spanner would lose its quorum and would become unavailable, at least for guaranteed-consistent operations. In practice, a network partition bad enough to bring down something like Spanner is pretty rare. (I worked at Google for four years, and I can't recall such a partition occurring.) But in theory it's certainly possible.
  - shin_lao 14y ago
    The CAP theorem states that at one point in time you can only have two in three. It doesn't say much more than the obvious. Obviously, if one node parts, you're either inconsistent or available.
    - Dave_Rosenthal 14y ago
      Just to clarify what the CAP theorem says: If one node parts, then that node is either inconsistent or not available. A fault-tolerant database could potentially stay up with a single unavailable node.
duaneb 14y ago
> It's interesting to see that the creators of BigTable and the early proponents of eventual consistency have invested the last 4.5 years building a system that adds back strong consistency guarantees. I think it's a large misconception that these are competing ideas. They're not. They just represent different use cases: search results don't need to be consistent within seconds of events. Customer information, however, does.
- Dave_Rosenthal 14y ago
  I agree that different projects prefer different tools, but these data stores most certainly do compete with each other for developer's use within Google. Per the paper: "At least 300 applications within Google use Megastore (despite its relatively low performance) because its data model is simpler to manage than Bigtable’s, and because of its support for synchronous replication across datacenters." So, the current trade off is ACID vs. performance, but I think the interesting point is that ACID is winning more and more now that a distributed ACID database is available to their developers. Again, according to Google: "We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions."
  - duaneb 14y ago
    That doesn't sound like competition; it sounds like they should have been using ACID all along. In any case, I don't think Google will stop using BigTable any time soon, especially with Spanner's massive latencies.
erichocean 14y ago
I'm with FoundationDB, which is a distributed NoSQL database with high performance cross-node ACID transactions. Which isn't at all comparable with Spanner, since Spanner works between clusters/datacenters, unlike FoundationDB. "Distributed" is getting severely overloaded when it comes to databases. We need to stop calling databases with sub-millisecond latency to other nodes "distributed". They're not distributed, they're clustered, and the techniques you can use with them are vastly different. Consistent transactions with a clustered database are a solved problem, full stop, and available to every developer with virtually any database backend, simply by building your transactions on top of Apache Zookeeper, and running whatever durable database you want underneath. Doing consistent distributed transactions at a global scale, which is what Spanner does, is so far novel. Doing ACID transactions in a 24-node cluster (what is described at FoundationDB's website, for instance) isn't impressive these days, and if that's all Google had done, you'd see a collective yawn around the web.

shin_lao 14y ago

From what I understand of this specific project, they "solved" the consistency problem (which is quite a feat, granted). They don't say they have an ACID NoSQL database, which is, to me, an oxymoron: ACIDity is useful if you have a powerful query language. In the end don't you fear that you might simply reinvent SQL? Or, am I missing something?

Evbn 14y ago
Which part of acid is relevant specifically to SQL (joins, I assume you mean)?
nlavezzo 14y ago
ACID and NoSQL are not mutually exclusive, although that is a common misconception due to the fact that almost all NoSQL databases gave up ACID in exchange for BASE (eventual consistency), making the development of their distributed databases much easier. However, ACID is simply a set of properties that allows you to group a set of reads & writes to a database into a transaction that is Atomic, Consistent, Isolated, and Durable. That set of reads & writes does not need to be SQL - in our case, it is a set of reads from and writes to a database of ordered keys and values. For example, you could read the values associated with keys "a" and "b" and "c" and based on that information, make a change to the values associated with keys "c" and "d" and then commit the transaction. The transaction then either fails completely (if one of the keys you read had been written to since your read, in which case you re-try the transaction) or succeeds completely. That is an ACID transaction in a (NoSQL) key-value store.
- shin_lao 14y ago
  That sounds useful, but not as useful as transactions on joins. It's great to have transactions when you have to cross reference data, on a key/value store, although you can have range queries I submit that kind of operations is less frequent. But still, it sure is great to be able to run a batch of operations with the confidence it will be transactional, I'm sure there are many use cases that can benefit from it. Oddly enough, it's strange that there aren't more NoSQL engines offering this feature as once you have MVCC you've done the hard part and AFAIK several NoSQL db have MVCC.
  - nlavezzo 14y ago
    Yep, those types of operations are less frequent in k/v stores, but we believe it is because of the lack of ACID in most of them. When you have ACID and ordering you can use transactions to keep data and multiple indexes on that data in synch, and use range operations to query the indexes. You can build all manner of higher level data structures with a transactional ordered k/v store, which is why our concept of "layers" is possible: www.foundationdb.com/#layers
damian2000 14y ago
It seems to me more like they have "solved" (or broken) the CAP theorem ... http://en.wikipedia.org/wiki/CAP_theorem http://en.wikipedia.org/wiki/CAP_theorem * Consistency (all nodes see the same data at the same time) * Availability (a guarantee that every request receives a response about whether it was successful or failed) * Partition tolerance (the system continues to operate despite arbitrary message loss or failure of part of the system) According to the theorem, a distributed system can satisfy any two of these guarantees at the same time, but not all three.
- snewman 14y ago
  No, they haven't broken the CAP theorem. If the network were to partition badly enough, Spanner would lose its quorum and would become unavailable, at least for guaranteed-consistent operations. In practice, a network partition bad enough to bring down something like Spanner is pretty rare. (I worked at Google for four years, and I can't recall such a partition occurring.) But in theory it's certainly possible.
- shin_lao 14y ago
  The CAP theorem states that at one point in time you can only have two in three. It doesn't say much more than the obvious. Obviously, if one node parts, you're either inconsistent or available.
  - Dave_Rosenthal 14y ago
    Just to clarify what the CAP theorem says: If one node parts, then that node is either inconsistent or not available. A fault-tolerant database could potentially stay up with a single unavailable node.

duaneb 14y ago

> It's interesting to see that the creators of BigTable and the early proponents of eventual consistency have invested the last 4.5 years building a system that adds back strong consistency guarantees. I think it's a large misconception that these are competing ideas. They're not. They just represent different use cases: search results don't need to be consistent within seconds of events. Customer information, however, does.

Dave_Rosenthal 14y ago
I agree that different projects prefer different tools, but these data stores most certainly do compete with each other for developer's use within Google. Per the paper: "At least 300 applications within Google use Megastore (despite its relatively low performance) because its data model is simpler to manage than Bigtable’s, and because of its support for synchronous replication across datacenters." So, the current trade off is ACID vs. performance, but I think the interesting point is that ACID is winning more and more now that a distributed ACID database is available to their developers. Again, according to Google: "We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions."
- duaneb 14y ago
  That doesn't sound like competition; it sounds like they should have been using ACID all along. In any case, I don't think Google will stop using BigTable any time soon, especially with Spanner's massive latencies.

erichocean 14y ago

I'm with FoundationDB, which is a distributed NoSQL database with high performance cross-node ACID transactions. Which isn't at all comparable with Spanner, since Spanner works between clusters/datacenters, unlike FoundationDB. "Distributed" is getting severely overloaded when it comes to databases. We need to stop calling databases with sub-millisecond latency to other nodes "distributed". They're not distributed, they're clustered, and the techniques you can use with them are vastly different. Consistent transactions with a clustered database are a solved problem, full stop, and available to every developer with virtually any database backend, simply by building your transactions on top of Apache Zookeeper, and running whatever durable database you want underneath. Doing consistent distributed transactions at a global scale, which is what Spanner does, is so far novel. Doing ACID transactions in a 24-node cluster (what is described at FoundationDB's website, for instance) isn't impressive these days, and if that's all Google had done, you'd see a collective yawn around the web.