4 ms·
> Today, we’re excited to announce the public beta for Cloud Spanner, a globally distributed relational database service that lets customers have their cake and
by koolba 10y ago
> Today, we’re excited to announce the public beta for Cloud Spanner, a globally distributed relational database service that lets customers have their cake and eat it too: ACID transactions and SQL semantics, without giving up horizontal scaling and high availability.
This is a bold claim. What do they know about the CAP theorem that I don't?
Separately, (emphasis mine):
> If you have a MySQL or PostgreSQL system that's bursting at the seams, or are struggling with hand-rolled transactions on top of an eventually-consistent database, Cloud Spanner could be the solution you're looking for. Visit the Cloud Spanner page to learn more and get started building applications on our next-generation database service.
From the rest of the article it seems like the wire protocol for accessing it is MySQL. I wonder if they mean to add a PostgreSQL compatibility layer at some point.
- teraflop 10y ago"High availability" is not the same thing as the "A" in the CAP theorem. Spanner chooses consistency over availability, but it is a HA system in the sense that it can tolerate datacenter outages.
- dsp1234 10y agoWhat do they know about the CAP theorem that I don't? I don't know your expertise on the subject, but they do have a post on this topic. Some highlights: "Does this mean that Spanner is a CA system as defined by CAP? The short answer is “no” technically, but “yes” in effect and its users can and do assume CA." "The purist answer is “no” because partitions can happen and in fact have happened at Google, and during some partitions, Spanner chooses C and forfeits A. It is technically a CP system." https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Sp...
- deleted 10y ago[deleted]
- larsiusprime 10y ago> What do they know about the CAP theorem that I don't? Their statement, for what it's worth: https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Sp...
- deleted 10y ago[deleted]
- jjoergensen 10y agoThey write that it's a CP system. So A is not guaranteed, but in practice they are able to provide A most of the time, with a private network and only 1 failure in all of 2016 (whatever that means).
- flyingramen 10y agoEric Brewer who is the author of the CAP theorem works at Google now. He has a post here: https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Sp....
- elvinyung 10y agoWhy? Strong consistency isn't mutually exclusive with scalability. Google has written about it at length[1][2][3]. Furthermore, there are already more than a few attempts underway to build scalable relational databases ("NewSQL") outside Google.[4] 1: https://research.google.com/pubs/pub36971.html https://research.google.com/pubs/pub36971.html 2: https://research.google.com/archive/spanner.html https://research.google.com/archive/spanner.html 3: http://datascienceassn.org/sites/default/files/F1%20A%20Distributed%20SQL%20Database%20That%20Scales.pdf http://datascienceassn.org/sites/default/files/F1%20A%20Dist... 4: http://db.cs.cmu.edu/papers/2016/pavlo-newsql-sigmodrec2016.pdf http://db.cs.cmu.edu/papers/2016/pavlo-newsql-sigmodrec2016....
- gigatexal 10y agoFirst time I'm hearing about newSQL thanks for these links.
- AsyncAwait 10y agoYou may be interested in CockroachDB[1] and TIDB[2], which are open-source newSQL databases inspired by Spanner and F1. 1 - https://www.cockroachlabs.com https://www.cockroachlabs.com 2 - https://github.com/pingcap/tidb https://github.com/pingcap/tidb
- sproctor 10y ago[disclosure: I'm the CTO at NuoDB, one of the NewSQL systems discussed in [4] above] Totally agree with your comment. While the "NewSQL" category is pretty broad and hard to define cleanly, one common theme is understanding the trade-offs that matter to your applications & operators as you distribute & scale SQL services. For instance, with NuoDB we've decided that standard ANSI SQL with complete DML/DDL support is a must-have for migrating existing applications. As you distribute the database, the ACID contract you expect with those tools also has to be maintained, and we do that. As others have noted in this thread, however, there are many variants of "consistency." I believe if you're trying to exploit distributed scale, and low-latency through locality of reference, then you're probably willing to trade-off serializability. Using MVCC, logical timestamps (not globally coordinated clocks) and a few other well-understood techniques we provide a scale-out solution that maintains standard levels of isolation and consistency. In practice our customers are traditional enterprises using these capabilities to migrate legacy, high-value systems into modern architectures. What Google has done is something very few other organizations can do: exploit their massive scale and hardware capabilities to offer something unique. From that perspective, they've also chosen to make trade-offs around syntax, indexing, transaction models etc. There's good commentary from my peer at Cockroach about this already on this thread, but the point is that Google picked a view of the database that makes sense for some applications and not others. Personally, I'm excited that Google has opened up this service and I'm sure it will lead to a lot of good introspection on what we need from modern data management services. For those who haven't read it yet, I'd highly recommend the Quizlet blog referenced elsewhere on this thread. It's a really good discussion about some trade-offs that Cloud Spanner has made, and why they would or wouldn't map well a given application. Bottom-line: both Cloud Spanner and NuoDB are examples of the modern, distributed, scale-out systems we're moving towards that focus on consistency, fault-tolerance and modernizing traditional SQL applications.
- awinograd 10y ago> To understand how, read this post by the author of the CAP Theorem and Google Vice President of Infrastructure, Eric Brewer. Seems like they might know a lot :)
- fmihaila 10y ago> This is a bold claim. What do they know about the CAP theorem that I don't? It's right there in the article: "Remarkably, Cloud Spanner achieves this combination of features without violating the CAP Theorem. To understand how, read this post by the author of the CAP Theorem and Google Vice President of Infrastructure, Eric Brewer." The post they are referring to: https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Sp...
- marknadal 10y agoTheoretically it means they are giving up on being Partition Tolerant. There was a popular post a while ago about how the P can't be sacrificed. Because if it is... everything else will fail. Being Google they are probably prideful enough to think their servers could never have an outage. Which yes, I agree with you, that is a very scary claim.
- CobrastanJorji 10y agoThis is thoroughly wrong. Cloud Spanner sacrifices the "A", not the "P." The cool thing being accomplished here is that the sacrifice to the A is greatly reduced (five or more 9s). There are several documents on the subject linked right off that page and elsewhere in these same comments, like this one: https://cloud.google.com/spanner/docs/whitepapers/SpannerAndCap.pdf https://cloud.google.com/spanner/docs/whitepapers/SpannerAnd...
- xapata 10y agoThe "choose two" of the CAP theorem is a bit misleading. By creating a distributed system, you've chosen P. So it's really choose one: A or C.
- wsh91 10y agoTo wit: In terms of CAP, Spanner claims to be both consistent and highly available despite operating over a wide area, which many find surprising or even unlikely. The claim thus merits some discussion. Does this mean that Spanner is a CA system as defined by CAP? The short answer is “no” technically, but “yes” in effect and its users can and do assume CA. The purist answer is “no” because partitions can happen and in fact have happened at Google, and during some partitions, Spanner chooses C and forfeits A. It is technically a CP system. However, no system provides 100% availability, so the pragmatic question is whether or not Spanner delivers availability that is so high that most users don't worry about its outages. For example, given there are many sources of outages for an application, if Spanner is an insignificant contributor to its downtime, then users are correct to not worry about it. https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Sp... (written by Eric Brewer, who proposed CAP)
- ProblemFactory 10y agoThere is a link at the end of the article to a blog post on exactly this topic: https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Sp... > The purist answer is “no” because partitions can happen and in fact have happened at Google, and during some partitions, Spanner chooses C and forfeits A. It is technically a CP system. > However, no system provides 100% availability, so the pragmatic question is whether or not Spanner delivers availability that is so high that most users don't worry about its outages.
- deleted 10y ago[deleted]
- jaboutboul 10y agoTheir explanation is here: https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Spanner-and-the-CAP-Theorem.html?m=1 https://cloudplatform.googleblog.com/2017/02/inside-Cloud-Sp...
- caleblloyd 10y ago> From the rest of the article it seems like the wire protocol for accessing it is MySQL. I wonder if they mean to add a PostgreSQL compatibility layer at some point. It looks like the wire protocol is Protocol Buffers and client libraries will likely use GRPC: https://cloud.google.com/spanner/docs/reference/rpc/google.spanner.v1 https://cloud.google.com/spanner/docs/reference/rpc/google.s...
- takeda 10y ago> From the rest of the article it seems like the wire protocol for accessing it is MySQL. I wonder if they mean to add a PostgreSQL compatibility layer at some point. I doubt it, the spanner is not even offering full MySQL capabilities, so it's unlikely to to support any advanced PG SQL.
- dantiberian 10y ago> This is a bold claim. What do they know about the CAP theorem that I don't? Your's is a bold claim also :) What do you know about the CAP theorem that Eric Brewer doesn't?