3 ms·
I should start by disclosing that I work part time in the Oracle Database group, but - of course - my HN account is entirely my own, despite occasional mild shi
by mike_hearn 3mo ago
I should start by disclosing that I work part time in the Oracle Database group, but - of course - my HN account is entirely my own, despite occasional mild shilling. The article itself is shilling for PlanetScale so that seems OK.
The author - certainly not deliberately! - says some untrue things about relational databases. The most important one is this:
> To understand why sharding is a necessary part of scaling relational databases...
But sharding isn't a necessary part of scaling relational databases. It's really just a requirement of simple databases like Postgres and MySQL.
If you want a relational database cluster that really does make 768 servers look like one, then you want what Oracle calls RAC ("real application clusters"). Your cloud will probably rent you access to one under a name like Autonomous Database. Self hosted it may be called ExaData, which is a unified hardware/software "rent a rack" style offering. You may be surprised to discover that it's not much more expensive than many managed Postgres offerings.
RAC can scale a non-sharded relational database horizontally. That means all queries can access all data, all SQL features like sequences and joins work, any server can take part in transactions with any other and in general it looks exactly like a really big single machine would. In other words, it's a synchronous multi-write-master system.
RAC scales very well. 768 servers is well within reach as long as your query patterns scale too i.e. don't all contend on writing to one row. Behind the scenes it uses a dedicated high speed RDMA network with lock coordination to transfer data blocks directly between nodes, never hitting disk for memory that's already in the buffer cache.
Additionally RAC is fully HA and supports rolling upgrades of the cluster whilst live. You don't need any NLBs or routers either. The client drivers automatically discover and load balance between nodes without needing intermediaries, transactions can start on one node and fail over to another without applications noticing, and so on. There's plenty of opportunity for caching and replication. You can run asynchronously replicated failover clusters, run multiple clusters in a Raft-driven globally coordinated super-cluster and can deploy coherent read-through caches anywhere; the main clusters will inform them the moment data in them becomes stale.
In other words, it can do a lot.
If for some reason you do need sharding then that's also supported with features like automatic sharding key distribution to client drivers that transparently route queries correctly, but most apps don't need this.
In case you're wondering why I say all this, firstly, obviously, I have a financial conflict of interest. But the database hasn't driven Oracle's stock price for a long time, so it's not a big one. These days it's all about cloud and AI.
No, the main reason is that HN fills up every month with blog posts where engineers talk about the incredible pain involved with scaling and running Postgres. And almost always, it's clear that they don't realize there's any alternative to that pain. It's not that they considered the options and then explain why they picked this one, it's that they think - as claimed in this article - that it's almost some fundamental limitation of reality itself, imposed by the laws of computer science.
There are lots of startups that lose time and money due to database problems they simply don't need to have. And that sucks. If they'd prefer to spend that time, pain and money to avoid using a DB from Oracle, fine, so be it. I won't argue with random devs about lawnmower memes. But if it's because they don't realize what's possible.... well, maybe someone will be helped by being aware of this. Database scaling problems are a choice, not an inevitability.
- marcosdumay 3mo agoLol about buying Oracle for performance! But anyway, do you have benchmarks from anybody not related to Oracle showing that performance? Because Oracle forbids talking about it...
- mike_hearn 3mo agoNot forbidden. You can email a specific address to share results/setup and get permission to publish if you want. Other commercial databases also do that, because there's so many ways to misconfigure a database to make it slow and competitors are strongly incentivized to do so. The question is what you'd want to benchmark. For example, imagine testing Postgres with a write load that goes well beyond what a single machine can do. It would collapse and query latencies would go through the roof. A horizontally scaled DB would keep going and process all those queries. Would you accept this as evidence that Postgres is slow or would you say it's not valid to benchmark at traffic loads Postgres physically cannot handle, given it never claimed to scale horizontally? Stuff like this is where benchmarking gets complicated. I used to work at a different company that sold a kind of database system. It was much faster than our nearest competitor, so we were surprised when that competitor claimed to an important customer they were just as fast as us. Their benchmark counted transactions that failed and rolled back due to overload (optimistic concurrency) as "successful".
- inigyou 3mo agoI suppose Oracle will only allow publication if your benchmark results look good. Can you tell us about a time you did a benchmark that made Oracle look bad, and they still let you publish it?
- marcosdumay 3mo ago> Other commercial databases also do that That's why you won't see many serious comments bragging about the performance of SQL Server either. Even though is has way fewer performance land-mines that you must design your entire architecture around than Oracle. Anyway, you are correct that RAC is a very impressive piece of software. If Postgres had something like it, it would be a beast. It's not enough to save Oracle, though.