5 ms·
And for many projects, Postgres is still cheaper than both. Having used both, I would much, much rather do the work to fit my project in Postgres/CockroachDB th
by nameless912 3y ago
And for many projects, Postgres is still cheaper than both. Having used both, I would much, much rather do the work to fit my project in Postgres/CockroachDB than use either Spanner or DynamoDB, which have WAY more footguns. Not to mention sudden cost spikes, vendor lock in, and god knows what else.
AWS and GCP (and Azure, and Oracle cloud, and bare Kubernetes via an operator, and...) support Postgres really well. Just...use Postgres.
- teaearlgraycold 3y agoPostgres is the database God himself would have made.
- nameless912 3y agoLike, I don't want to fanboy, but it's _hard_ to find a use case that postgres can't handle at small to medium (i.e. 90% of projects) scale.
- 0xbadcafebee 3y agoIt's hard to find a use case that a plain old filesystem can't handle at small to medium scale. But there are perhaps more important considerations than just "can it handle it"
- diordiderot 3y agoLike what? (Genuinely curious)
- pier25 3y agoProbably more like 99%
- nvm0n2 3y agoUh let's not get carried away. It's fine with enough work maybe. But Postgres has a lot of awkwardness too. HA is a pain, major version upgrades are a pain, JS or JVM stored procs are a pain, configuring auth is a pain. There is a reason so many people are desperate to pay someone else to run Postgres for them instead of just renting a few VMs and doing it themselves.
- hot_gril 3y agoIt's the best, but it's far from perfect. Default mode is non-ACID, and going `serializable` mode makes it very slow. Spanner is always ACID... but always slow.
- paxys 3y agoPostgres is a piece of software. Cloud Spanner/Dynamo etc are managed services. It makes no sense to directly compare one with the other.
- esafak 3y agoRead "Cloud SQL for PostgreSQL"
- nameless912 3y ago> AWS and GCP (and Azure, and Oracle cloud, and bare Kubernetes via an operator, and...) support Postgres really well. I think they're directly comparable with this context.
- hatsix 3y agoSure, You can compare Cloud SQL vs Cloud Spanner and RDS vs Dynamo, but it makes more sense to just say "Postgres" and assume that the reader can figure out that it means "Whatever managed postgres service you want to use". The entire point is that every cloud provider has a managed postgres offering, and there's no vendor lock-in. Though, technically, Dynamo does have a docker image you could run in other cloud providers if it came down to that, you'd get no support for it.
- res0nat0r 3y agoI don't think it's really relevant to compare plain Postgres to Spanner, most folks have no need for something like this. It is made for folks who need to do millions of ACID type transactions a second/minute from all over the globe, and have a globally consistent database at all times. There's a reason why Google installed and built their own atomic clocks and put them in their datacenters, it is to facilitate global timekeeping for this type of services. Most likely 99.9% of the time this type of database is overkill, and also likely way more expensive than you need. https://cloud.google.com/spanner/docs/true-time-external-consistency https://cloud.google.com/spanner/docs/true-time-external-con...
- riku_iki 3y ago
- aketchum 3y agojust migrated off of PG to ddb as the main db for my application (still copying data to SQL for data analytics). Working with distributed functions and code hosted on lambdas, the connection management to SQL became a nightmare with dropped requests all over the place.
- nameless912 3y agoA connection pooling proxy should fix this shouldn't it? I think both Google and AWS have this solved for functions as a service.
- aketchum 3y agoyeah but you run into other issues when you put all of your lambdas on the same VPC
- nameless912 3y agoNot to be that guy, but why lambdas? I'm genuinely curious. I've never found the "cost savings" (big air quotes) worth it in comparison to the increased configuration/permissions complexity. Especially when Fargate exists, where you can just throw a docker container at AWS, what do Lambdas add? The zero scaling?
- sakopov 3y agoYour mileage may vary, but I think in majority of the cases Fargate is going to be significantly more expensive than Lambda.
- nameless912 3y agoBut at low request volumes, either is a rounding error to a medium size enterprise, and for personal projects IMO Lambda is a huge PITA.
- 3y ago
- emodendroket 3y agoI mean sure, NoSQL gives you more opportunities to screw stuff up because it's doing less for you. But it can be a reasonable tradeoff in some scenarios anyway
- alisonkisk 3y agoSpanner and Postgres both support both SQL and NoSQL
- bananapub 3y ago> And for many projects, Postgres is still cheaper than both. ok? and sqlite3 in memory is even cheaper than postgres! if you can use (and support correctly) postgres then you should use it, obviously there's no point using a globally scalable P-level database if you can just fit all your data on one machine with posthgres.
- airstrike 3y agoExcept for projects for which NoSQL is a better fit than a RDBMS, no? If I'm writing a chat app with millions of messages and very little in the way of "relationships", should I use Postgres or some flavor of NoSQL? Honest question.
- silisili 3y agoI've found it's pretty easy to massage data either way, depending on your preference. The one I'm working on now ultimately went from postgres, to mysql, to dynamo, the latter mainly for cost reasons. You do have to think about how to model the data in each system, but there are very few cases IMO where one is strictly 'better.'
- jpgvm 3y agoGolden Rule of data: Use PostgreSQL unless you have an extremely good reason not to. PostgreSQL is extremely good at append-mostly data, i.e like a chat log and has powerful partitioning features that allow you to keep said chat logs for quite some time (with some caveats) while keeping queries fast. Generally speaking though PostgreSQL has powerful features for pretty much every workload, hence the Golden Rule.
- GabeWeiss_ 3y ago100% this, and even though I work for Google I absolutely agree. BUT, for the folks that need it, PostgreSQL just DOESN'T cut it, so it's why we have databases like DynamoDB, Spanner, etc. Arguing that we should "Just use PG" is kinda a moot point.
- nameless912 3y agoI think I said this in another comment, but I'm not shitting on Spanner or DDB's right to exist here. Obviously, there are _some_ problems for which a globally distributed ACID compliant SQL-compatible database are useful. However, those problems are few and far between, and many/most of them exist at companies like Google. The fact is your average small to medium size enterprise doesn't need and doesn't benefit from DDB/Spanner, but "enterprise architects" love to push them for some ungodly reason.
- remus 3y agoI know the spanner marketing blurb says you can scale down etc. But I think in practice spanner is primarily aimed at use cases where you'd struggle to fit everything in a single postgres instance. Having said that I guess I broadly agree with your comment. It seems like a lot of people like to plan for massive scale while they have a handful of actual users.
- nameless912 3y agoI said this in another comment, but I have seen _two_ applications in my career that actually had a request load that might warrant something like one of these databases. One was an application with double digit million MAU and thousands of RPS on a very shardable data set, which fit Spanner's ideal access pattern and performance profile pretty well, but we paid an absolute arm and a leg for the privilege and ended up implementing a distributed cache in front of Spanner to reduce costs. The other just kept the data set in memory and flushed to disk/S3 backup periodically because in that case liveness was more important than completeness. In the first case, the database created as many problems as it solved (which is true of any large application running at scale; your data store will _always_ be suboptimal). A fancy, expensive NoSQL database won't save you from solving hard engineering problems. At smaller scales (on the order of tens-hundreds of RPS), it's hard to go wrong with any established SQL (or open source NoSQL if that floats your boat) database, and IMO Postgres is the most stable and best bang for your engineering buck feature wise.
- joshuamorton 3y agoSpanner isn't NoSQL. It's schema'd.
- ndriscoll 3y agoPostgres/mysql shouldn't have much trouble doing thousands of RPS on a laptop for basic CRUD queries (i.e. as long as you don't need to do table scans or large index range scans). It's possible to squeeze a lot more than that out of them.
- hot_gril 3y agoMy team bought the "scale down" thing and got bit. Using Spanner is giving up a lot for the scalability, and if you ever reach the scale where a single node DB doesn't make sense anymore, I don't know if Spanner is still the answer, let alone Spanner with your old design still intact. For one, Postgres has scaling options like Citus. Or maybe you don't need a scalable DB even at scale, cause you shard at a higher layer instead.
- 0xbadcafebee 3y agoPostgres and Spanner do different things, in different ways, with different costs, risks, and implications. You could "just use" anything that is completely different and slightly cheaper. You could use a GitHub repository and just store your records as commits, for free, that's plenty cheap and works for small projects. But not really the same thing, is it?
- nameless912 3y agoMy point is that I've seen very, very few situations (I can think of two in my entire career so far) where a "hyperscale NoSQL database" was actually the right choice to solve the problem. I find that a lot of folks turn to these databases for imagined scale needs, not actual hard problems that need solving.
- eklitzke 3y agoSpanner has a SQL interface, fyi.
- nameless912 3y agoI don't think the API you interface with fundamentally changes the point that Spanner is hard to recommend from an engineering perspective at anything except the absolute most massive of scales, and even then it will create nearly as many problems as it solves. I'm not saying spanner is _wrong_ or shouldn't exist, but it's very difficult to be in the position where Spanner is the critical key to your application's success and not replaceable by <insert other, cheaper database here>.
- endorphine 3y agoCare to elaborate on what are the problems that it will create? Honest question.
- nameless912 3y ago
- supportengineer 3y agoIs there any company that hosts Postgres in the cloud (and does nothing else) and has great customer service?
- LVB 3y agoI've only kicked the tires, but https://neon.tech https://neon.tech is a pure hosted Postgres play. I'd be curious to hear if anyone has used them for a real projects, and how that went.
- M3t0r 3y agoGive https://aiven.io/ https://aiven.io/ a try. But know that I'm biased ;-) Also https://www.postgresql.org/support/professional_hosting/ https://www.postgresql.org/support/professional_hosting/
- folmar 3y agoElephantsql, I can't say much for the customer service as I never had any trouble.
- GabeWeiss_ 3y agoBut that's kind of a moot point. I mean, if you're even looking at the likes of DynamoDB or Spanner, it's because you need the scale of those engines. PostgreSQL is fantastic, and even working for Google, I 100% agree with you. Just use PG...until you can't. Once you're in the realm of Spanner and DynamoDB, that's where this discussion becomes more of a thing.
- silisili 3y agoNot necessarily true. DynamoDB on demand pricing is actually way cheaper than RDS or EC2 based anything for small workloads, especially when you want it replicated.
- GabeWeiss_ 3y agoFair enough, I'm not (obviously) as familiar with the AWS systems and how the price to performance ratio works out for all the products.
- agonz253 3y agoIt's not purely a matter of cost, right? Say you want or need a highly available, high performance distributed database with externally consistent semantics. Are you going to handle the sharding of your Postgres data yourself? What replication system will you use for each shard? How will you ensure strong consistency? Will you be able to do transactions across shards? These are problems that systems like Spanner, CockroachDB, etc solve for you.
- YetAnotherNick 3y agoJust curious, why would distributed be design requirement? Is individual machine failure likely in AWS/GCP? The only failure I have seen in region level issues which spanner or dynamo don't help with AFAIK.
- agonz253 3y agoIndividual machine failure is not likely, but we're hypothesizing the need for multiple shards for high performance. So now we have more machines and so the probability of failure increases. So we need to add replication, but then we need to deal with data getting out of sync, etc.... As others have mentioned though, these issues only really become important at a certain scale.