4 ms·

How does Spanner stay competitive with other NewSQL databases? Doesn't the pricing and the API lock-in mean that people will just consider open-source NewSQL da

How does Spanner stay competitive with other NewSQL databases? Doesn't the pricing and the API lock-in mean that people will just consider open-source NewSQL databases first? (Not a shill for any company, honestly just curious.)

donaldihunter 9y ago
Spanner is Google's private in-house database so it's not competing with NewSQL vendors. Edit: Late to the news that Google is selling its spanner warez. Well it's been in the game longer than anyone else and is battle tested by Google.
- elvinyung 9y ago
  > so it's not competing with NewSQL vendors. But it... literally is? https://cloud.google.com/spanner/ https://cloud.google.com/spanner/
- brucephillips 9y ago
  Spanner is not private.
brucephillips 9y ago
Every database has API lock-in.
- elvinyung 9y ago
  I would say that Spanner is "more" lock-in than, say, an open-source database that's mostly wire-compatible with PostgreSQL. But even then, the other big thing you have to compete on is pricing, right?
  - brucephillips 9y ago
    Like any othet SaaS product, you're paying for management, which is valuable.
jitl 9y ago
I'm much more likely to use Spanner than I am to operate my own installation of something like CockroachDB. To operate a database: 1. I need to provision N nodes across regions and availability zones 2. I need to patch those nodes 3. I need to back those nodes up 4. I need to build automated systems to rotate those nodes if they become unhealthy 5. I need to build automated systems to restore my entire database from backups in event of disaster With Spanner, I don't have to do any of that, really. Maybe a bit of fiddling with replicas or something.
- deleted 9y ago
  [deleted]
rollcat 9y ago
My guess is that the operational complexity of running a globally-distributed, fault-tolerant, performant SQL database surpasses the Spanner monthly bill (currently $0.90/h, or ~$675/mo, for a single node; $3/h or $2250/mo for a multi-region instance). Definitely not a side project kind of money, but think of the engineer-hours you'd put into maintaining such a setup in production -- plus the hardware costs.
- berns 9y ago
  Also note that a minimum of 3 nodes is recommended for production environments. So it's $2000/month at least to get started.
  - ibejoeb 9y ago
    That's still a fraction of a competent database person, so it's still a deal. The downside is its limited feature set. There's no fixed-precision decimal type, and it's a pretty old SQL implementation.
gervase 9y ago
For many businesses, pricing is much less important than business continuity, in the sense that they are willing to pay more in the interests of getting "the name brand", so to speak. Why would they risk another RethinkDB by choosing a startup when they can go with Google's dogfooded version? (This assumes they're already interested in outsourcing their databases in the first place.)
carterehsmith 9y ago
>> How does Spanner stay competitive with other NewSQL databases? Good question, but... what other NewSQL databases we are talking about here? That is, NewSQL databases that offer strong consistency, and multi-region replication. I do not see many of them.
mdasen 9y ago
It's really hard to compare database performance, but it looks like Spanner probably is at a pretty competitive price-performance point. Google advertises that a $0.90/hour node can do 10,000 reads per second and 2,000 writes per second. Amazon wrote a blog post comparing Aurora and PostgreSQL performance and got ~38,000 TPS on Aurora (vs. ~18,000 on PostgreSQL) on a m4.16xlarge, but that would cost $9.28/hour (https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgresql-compatibility/ https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgr...). Given that you could make a 10-node cluster for that price with 100,000 reads/second, it seems to be reasonably price competitive. Even if you were going to run instances yourself, a 16xlarge costs $4.256/hour and 4 spanner nodes seem like they'd be competitive with PostgreSQL (or even Aurora) on that hardware. When you're building your business, you make trade-offs. Spanner means high-uptime and throughput without worrying about ops and scaling. At my company, we run our own database clusters, but we also have teams of people working on them (Spanner and RDS didn't exist when the company started). Do you want to have to worry about how you'll partition data and make sure everything stays consistent? Do you want to have to write more complex code to accommodate for less consistency? That will mean you're slower to ship features and those features will be buggier. Do you want to hit scaling limits where there isn't a larger EC2 instance to put your database on? Do you want to have to deal with growing those instances? CockroachDB is meant to handle many of these things, but it's young and you're still going to need to deal with running it. Databases (at scale) can provide all sorts of subtle trouble. For example, in one of our clusters, we lost a node and the cluster decided to rebalance things. But it didn't have the IO capacity to rebalance and started grinding to a halt. This was with a very widely used NoSQL/NewSQL store. Do you like tracing and heap dumps? The issue with running your own cluster isn't when things are going well, but when things start going sideways.

donaldihunter 9y ago

Spanner is Google's private in-house database so it's not competing with NewSQL vendors. Edit: Late to the news that Google is selling its spanner warez. Well it's been in the game longer than anyone else and is battle tested by Google.

elvinyung 9y ago
> so it's not competing with NewSQL vendors. But it... literally is? https://cloud.google.com/spanner/ https://cloud.google.com/spanner/
brucephillips 9y ago
Spanner is not private.

brucephillips 9y ago

Every database has API lock-in.

elvinyung 9y ago
I would say that Spanner is "more" lock-in than, say, an open-source database that's mostly wire-compatible with PostgreSQL. But even then, the other big thing you have to compete on is pricing, right?
- brucephillips 9y ago
  Like any othet SaaS product, you're paying for management, which is valuable.

jitl 9y ago

I'm much more likely to use Spanner than I am to operate my own installation of something like CockroachDB. To operate a database: 1. I need to provision N nodes across regions and availability zones 2. I need to patch those nodes 3. I need to back those nodes up 4. I need to build automated systems to rotate those nodes if they become unhealthy 5. I need to build automated systems to restore my entire database from backups in event of disaster With Spanner, I don't have to do any of that, really. Maybe a bit of fiddling with replicas or something.

deleted 9y ago
[deleted]

rollcat 9y ago

My guess is that the operational complexity of running a globally-distributed, fault-tolerant, performant SQL database surpasses the Spanner monthly bill (currently $0.90/h, or ~$675/mo, for a single node; $3/h or $2250/mo for a multi-region instance). Definitely not a side project kind of money, but think of the engineer-hours you'd put into maintaining such a setup in production -- plus the hardware costs.

berns 9y ago
Also note that a minimum of 3 nodes is recommended for production environments. So it's $2000/month at least to get started.
- ibejoeb 9y ago
  That's still a fraction of a competent database person, so it's still a deal. The downside is its limited feature set. There's no fixed-precision decimal type, and it's a pretty old SQL implementation.

gervase 9y ago

For many businesses, pricing is much less important than business continuity, in the sense that they are willing to pay more in the interests of getting "the name brand", so to speak. Why would they risk another RethinkDB by choosing a startup when they can go with Google's dogfooded version? (This assumes they're already interested in outsourcing their databases in the first place.)

carterehsmith 9y ago

>> How does Spanner stay competitive with other NewSQL databases? Good question, but... what other NewSQL databases we are talking about here? That is, NewSQL databases that offer strong consistency, and multi-region replication. I do not see many of them.

mdasen 9y ago

It's really hard to compare database performance, but it looks like Spanner probably is at a pretty competitive price-performance point. Google advertises that a $0.90/hour node can do 10,000 reads per second and 2,000 writes per second. Amazon wrote a blog post comparing Aurora and PostgreSQL performance and got ~38,000 TPS on Aurora (vs. ~18,000 on PostgreSQL) on a m4.16xlarge, but that would cost $9.28/hour (https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgresql-compatibility/ https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgr...). Given that you could make a 10-node cluster for that price with 100,000 reads/second, it seems to be reasonably price competitive. Even if you were going to run instances yourself, a 16xlarge costs $4.256/hour and 4 spanner nodes seem like they'd be competitive with PostgreSQL (or even Aurora) on that hardware. When you're building your business, you make trade-offs. Spanner means high-uptime and throughput without worrying about ops and scaling. At my company, we run our own database clusters, but we also have teams of people working on them (Spanner and RDS didn't exist when the company started). Do you want to have to worry about how you'll partition data and make sure everything stays consistent? Do you want to have to write more complex code to accommodate for less consistency? That will mean you're slower to ship features and those features will be buggier. Do you want to hit scaling limits where there isn't a larger EC2 instance to put your database on? Do you want to have to deal with growing those instances? CockroachDB is meant to handle many of these things, but it's young and you're still going to need to deal with running it. Databases (at scale) can provide all sorts of subtle trouble. For example, in one of our clusters, we lost a node and the cluster decided to rebalance things. But it didn't have the IO capacity to rebalance and started grinding to a halt. This was with a very widely used NoSQL/NewSQL store. Do you like tracing and heap dumps? The issue with running your own cluster isn't when things are going well, but when things start going sideways.