5 ms·

Coming from Google, where Spanner is this magical technology that supports infinite horizontal sharding with transactions and has become the standard storage en

Coming from Google, where Spanner is this magical technology that supports infinite horizontal sharding with transactions and has become the standard storage engine for everything at Google (almost every project not using Spanner was moving to Spanner), I'm curious how Figma evaluated Cloud Spanner. Cloud Spanner does have a postgres translation layer, though I don't know how well it works. It seems like they've (hopefully only temporarily) given up real transactional support with their horizontal postgres scheme?

echelon 3y ago
Why would anyone marry themselves to Google? That sounds like the most boneheaded move possible. First, you should never be beholden to a single vendor for the most critical technological underpinnings. You're backing yourself into a corner. But more importantly, Google can't even figure out how to prioritize their cloud efforts. That's not a good partnership to be in for anyone except Google. I wouldn't care if my solution was 10x worse than Cloud Spanner from a technology perspective. It'd be 1000x better from a strategy perspective. You can hire engineers to do consistency at scale. It's your core competency and you can't just handwave and outsource that, lest you wind up stuck in a crevasse. Hire smart engineers and do the work yourself. It'll pay off.
- Thaxll 3y ago
  You can't, that's why spanner only exists a Google. That tech is that good, not found anywhere else.
  - echelon 3y ago
    But you don't need it. There are a limitless number of ways to deal with the problems spanner solves. And by choosing your own, you can focus on the subset that matter to your case and not be tied down to Google.
hipadev23 3y ago
The problem is nobody outside Google trusts them to run or operate anything. Edit: To the Googlers downvoting these comments. Your behavior only reinforces our views.
- szundi 3y ago
  Google means: good chance discontinued after you have worked out the bugs and have a stable system at last
  - groestl 3y ago
    This is definitely not the case with their core cloud products. > almost every project not using Spanner was moving to Spanner This even includes Datastore. Even Datastore moved to Spanner.
ksb 3y ago
Ping time from AWS data centers to GCP ones
umvi 3y ago
Never a good idea to rely on Google proprietary tech (unless you are Google)... it could be sunset at any time without warning. I use GCP but I try my best to stay Google agnostic (avoid GCP-only offerings, etc) so that I can move to AWS if Google pulls the rug out from under me.
- rockostrich 3y ago
  GCP products have a much better track record than Google consumer products when it comes to support since there are usually enterprise customers with multi-year contracts worth tens, if not hundreds, of millions of dollars using them.
  - callalex 3y ago
    That’s what AppEngine customers thought.
    - weitendorf 3y ago
      I worked on AppEngine (well, Serverless Compute) at Google for over 4 years and left on Friday. Did something happen to AppEngine in the last week?
      - callalex 3y ago
        I’m talking about the pricing disaster that happened in 2017/2018 where user prices went up from 10x-100x because Google wanted to kill the product without actually killing it.
    - tempnow987 3y ago
      Didn't they also hammer folks using google maps in a business / cloud context? I was also an old app engine user, but bailed ages ago (original app engine).
  - jerrygenser 3y ago
    IoT is one example of a big backbone service that was sunset.
shalabhc 3y ago
Global consistency is expensive, both latency-wise and cost-wise. In reality most apps don't need global serializability across all objects. For instance, you probably don't need serializability across different tenants, organizations, workspaces, etc. Spanner provides serializability across all objects IIUC - so you pay for it whether you need it or not. The other side of something like Spanner is the quorum-based latency is often optimized by adding another cache on top, which instantly defeats the original consistency guarantees. The consistency of (spanner+my_cache) is not the same as the consistency of spanner. So if we're back to app level consistency guarantees anyway, turns out the "managed" solution is only partial. Ideally the managed db systems would have flexible consistency, allowing me to configure not just which object sets need consistency but also letting me configure caches with lag tolerance. This would let me choose trade-offs without having to implement consistent caching and other optimization tricks on top of globally consistent/serializable databases.
- eatonphil 3y ago
  See also: "Strict-serializability, but at what cost, for what purpose?" https://muratbuffalo.blogspot.com/2022/08/strict-serializability-but-at-what-cost.html https://muratbuffalo.blogspot.com/2022/08/strict-serializabi....
- foota 3y ago
  While it doesn't help much with the costs of replication, Spanner can be configured with read only replicas that don't participate in voting for commits, so they don't impact the quorum latency. Reads can then be done with different consistency requirements, e.g., bounded staleness (which guarantees data less stale than the time bound requested). See https://cloud.google.com/spanner/docs/reference/rest/v1/TransactionOptions https://cloud.google.com/spanner/docs/reference/rest/v1/Tran... or https://cloud.google.com/spanner/docs/reads#read_types https://cloud.google.com/spanner/docs/reads#read_types and https://cloud.google.com/spanner/docs/create-manage-configurations#create-configuration https://cloud.google.com/spanner/docs/create-manage-configur...
opportune 3y ago
My perspective from working both inside and outside of Google: The external spanner documentation doesn’t seem as good as the internal documentation, in my opinion. Because it’s not generally well known outside of G, they ought to do a better job explaining it and its benefits. It truly is magical technology but you have to be a database nerd to see why. It’s also pretty expensive and because you generally need to rewrite your applications to work with it, there is a degree of lockin. So taking on Spanner is a risky proposition - if your prices get hiked or it starts costing more than you want, you’ll have to spend even more time and money migrating off it. Spanner’s advantages over other DBs (trying to “solve” the CAP theorem) then become a curse, because it’s hard to find any other DB that gives you horizontal scaling, ACID, and high availability out of the box, and you might have to solve those problems yourself/redesign the rest of your system. Personally I would consider using Cloud Spanner, but I wouldn’t bet my business on it.
- nojvek 3y ago
  If you really have that much data and traffic, the $ costs start to add up to multiple engineer comp costs. At that point it’s cheaper to move to something you have good control over. I.e sharding at application layer and connecting to the DB instance replica where the customer data is hosted.
  - weitendorf 3y ago
    Depends. The cost may pay for itself but the engineers you have already may have higher ROI things to do. It's also nice to have operational stuff managed for you. Personally I'd be happy to pay extra for the kinds of problems Spanner solves to free myself up to do other things (to a point, ofc). > sharding at application layer and connecting to the DB instance replica where the customer data is hosted. Spanner does global consistency/replication. If having good performance per-tenant globally is a concern, this helps a lot, and is hard to implement on your own. It can also ultimately save you money by limiting cross-region traffic.
zenbowman 3y ago
I wouldn't say "infinite", its still susceptible to read hotspotting; and while fine-grained locking enables generally higher write throughputs, you can still get in a situation where interconnected updates end up being pretty slow. That said, its way better than anything else I've used in my career.

echelon 3y ago

Why would anyone marry themselves to Google? That sounds like the most boneheaded move possible. First, you should never be beholden to a single vendor for the most critical technological underpinnings. You're backing yourself into a corner. But more importantly, Google can't even figure out how to prioritize their cloud efforts. That's not a good partnership to be in for anyone except Google. I wouldn't care if my solution was 10x worse than Cloud Spanner from a technology perspective. It'd be 1000x better from a strategy perspective. You can hire engineers to do consistency at scale. It's your core competency and you can't just handwave and outsource that, lest you wind up stuck in a crevasse. Hire smart engineers and do the work yourself. It'll pay off.

Thaxll 3y ago
You can't, that's why spanner only exists a Google. That tech is that good, not found anywhere else.
- echelon 3y ago
  But you don't need it. There are a limitless number of ways to deal with the problems spanner solves. And by choosing your own, you can focus on the subset that matter to your case and not be tied down to Google.

hipadev23 3y ago

The problem is nobody outside Google trusts them to run or operate anything. Edit: To the Googlers downvoting these comments. Your behavior only reinforces our views.

szundi 3y ago
Google means: good chance discontinued after you have worked out the bugs and have a stable system at last
- groestl 3y ago
  This is definitely not the case with their core cloud products. > almost every project not using Spanner was moving to Spanner This even includes Datastore. Even Datastore moved to Spanner.

ksb 3y ago

Ping time from AWS data centers to GCP ones

umvi 3y ago

Never a good idea to rely on Google proprietary tech (unless you are Google)... it could be sunset at any time without warning. I use GCP but I try my best to stay Google agnostic (avoid GCP-only offerings, etc) so that I can move to AWS if Google pulls the rug out from under me.

rockostrich 3y ago
GCP products have a much better track record than Google consumer products when it comes to support since there are usually enterprise customers with multi-year contracts worth tens, if not hundreds, of millions of dollars using them.
- callalex 3y ago
  That’s what AppEngine customers thought.
  - weitendorf 3y ago
    I worked on AppEngine (well, Serverless Compute) at Google for over 4 years and left on Friday. Did something happen to AppEngine in the last week?
    - callalex 3y ago
      I’m talking about the pricing disaster that happened in 2017/2018 where user prices went up from 10x-100x because Google wanted to kill the product without actually killing it.
  - tempnow987 3y ago
    Didn't they also hammer folks using google maps in a business / cloud context? I was also an old app engine user, but bailed ages ago (original app engine).
- jerrygenser 3y ago
  IoT is one example of a big backbone service that was sunset.

shalabhc 3y ago

Global consistency is expensive, both latency-wise and cost-wise. In reality most apps don't need global serializability across all objects. For instance, you probably don't need serializability across different tenants, organizations, workspaces, etc. Spanner provides serializability across all objects IIUC - so you pay for it whether you need it or not. The other side of something like Spanner is the quorum-based latency is often optimized by adding another cache on top, which instantly defeats the original consistency guarantees. The consistency of (spanner+my_cache) is not the same as the consistency of spanner. So if we're back to app level consistency guarantees anyway, turns out the "managed" solution is only partial. Ideally the managed db systems would have flexible consistency, allowing me to configure not just which object sets need consistency but also letting me configure caches with lag tolerance. This would let me choose trade-offs without having to implement consistent caching and other optimization tricks on top of globally consistent/serializable databases.

eatonphil 3y ago
See also: "Strict-serializability, but at what cost, for what purpose?" https://muratbuffalo.blogspot.com/2022/08/strict-serializability-but-at-what-cost.html https://muratbuffalo.blogspot.com/2022/08/strict-serializabi....
foota 3y ago
While it doesn't help much with the costs of replication, Spanner can be configured with read only replicas that don't participate in voting for commits, so they don't impact the quorum latency. Reads can then be done with different consistency requirements, e.g., bounded staleness (which guarantees data less stale than the time bound requested). See https://cloud.google.com/spanner/docs/reference/rest/v1/TransactionOptions https://cloud.google.com/spanner/docs/reference/rest/v1/Tran... or https://cloud.google.com/spanner/docs/reads#read_types https://cloud.google.com/spanner/docs/reads#read_types and https://cloud.google.com/spanner/docs/create-manage-configurations#create-configuration https://cloud.google.com/spanner/docs/create-manage-configur...

opportune 3y ago

My perspective from working both inside and outside of Google: The external spanner documentation doesn’t seem as good as the internal documentation, in my opinion. Because it’s not generally well known outside of G, they ought to do a better job explaining it and its benefits. It truly is magical technology but you have to be a database nerd to see why. It’s also pretty expensive and because you generally need to rewrite your applications to work with it, there is a degree of lockin. So taking on Spanner is a risky proposition - if your prices get hiked or it starts costing more than you want, you’ll have to spend even more time and money migrating off it. Spanner’s advantages over other DBs (trying to “solve” the CAP theorem) then become a curse, because it’s hard to find any other DB that gives you horizontal scaling, ACID, and high availability out of the box, and you might have to solve those problems yourself/redesign the rest of your system. Personally I would consider using Cloud Spanner, but I wouldn’t bet my business on it.

nojvek 3y ago
If you really have that much data and traffic, the $ costs start to add up to multiple engineer comp costs. At that point it’s cheaper to move to something you have good control over. I.e sharding at application layer and connecting to the DB instance replica where the customer data is hosted.
- weitendorf 3y ago
  Depends. The cost may pay for itself but the engineers you have already may have higher ROI things to do. It's also nice to have operational stuff managed for you. Personally I'd be happy to pay extra for the kinds of problems Spanner solves to free myself up to do other things (to a point, ofc). > sharding at application layer and connecting to the DB instance replica where the customer data is hosted. Spanner does global consistency/replication. If having good performance per-tenant globally is a concern, this helps a lot, and is hard to implement on your own. It can also ultimately save you money by limiting cross-region traffic.

zenbowman 3y ago

I wouldn't say "infinite", its still susceptible to read hotspotting; and while fine-grained locking enables generally higher write throughputs, you can still get in a situation where interconnected updates end up being pretty slow. That said, its way better than anything else I've used in my career.